Measurement Image Formats

This guide explains how to control the encoding of the device images that Elements Hub returns inside a measurement (RAE) container, and how to decode the raw image payload from C++.

Background

When a measurement is triggered against an Aesthetix device, the resulting RAE container can include one or more device images — for example the scratch view, the surface view, the LED sparkle stack or the LED shade stack. By default these images are encoded as PNG. Callers can choose the encoding per measurement: PNG, JPEG, or the unaltered raw bytes that the device sent.

Selecting the encoding

The encoding is selected through the optional imageFormat operation parameter when triggering a measurement:

POST /v1/devices/{deviceId}/measurement-triggers
Content-Type: application/json

{
  "measurementKey": "measure",
  "containerFormat": "raeBinary",
  "parameters": [
    { "key": "metricGroupKeys", "value": ["..."] },
    { "key": "imageFormat",     "value": "image/x-raw" }
  ]
}

Unknown values are rejected with HTTP 400 and the error code E42InvalidImageFormat.

A small number of device images (surface, waviness, spot) are post-processed (cropped) on the hub side before being encoded. With image/png and image/jpeg you receive the cropped variant; with image/x-raw you receive the original, uncropped bytes as emitted by the device. Most images — including the shade images (ledShadeImages_*) — are not post-processed, so all three encodings cover identical pixel data.

Locating images in the container

The container exposes the device images as File components in the measurement tree. Array images such as the shade stack use one entry per slot with a numeric suffix:

ledShadeImages              (Array)
├── ledShadeImages_0        (File, MimeType=image/x-raw)
├── ledShadeImages_1        (File)
├── ledShadeImages_2        (File)
├── ledShadeImages_3        (File)
├── ledShadeImages_4        (File)
└── ledShadeImages_5        (File)

The BinaryData field of each File component holds the byte stream — either the encoded PNG/JPEG or the raw layout described in the next section, depending on the imageFormat you requested.

Layout of image/x-raw

Every raw image starts with a fixed 16-byte little-endian header, immediately followed by the pixel data:

+------------------------------------------+
| Header (16 bytes, little-endian)         |
+------------------------------------------+
|  Offset 0..3   : Height           (int32)|
|  Offset 4..7   : Width            (int32)|
|  Offset 8..11  : Channels         (int32)|
|  Offset 12..15 : BytesPerPixel    (int32)|
+------------------------------------------+
| Pixel data (Height * Stride bytes)       |
| Row-major, top-down                      |
+------------------------------------------+

• Endianness: little-endian for every header field.
• Stride (bytes per row): (Width * Channels * BytesPerPixel * 8 + 7) / 8 — byte-aligned, no row padding. For all current device images this is simply Width * Channels * BytesPerPixel.
• No padding between the header and the pixel data.

Pixel formats

The combination of Channels and BytesPerPixel describes the pixel layout:

No other combinations occur in current Aesthetix firmware.

Parse the format out of the header rather than hard-coding a format per image type. The device firmware may change which format it uses for a given image in future updates.

Parsing the raw stream in C++

Header & pixel view (endian-safe, no external dependencies)

#include <cstdint>
#include <cstring>
#include <stdexcept>

struct AesthetixRawImage {
    int32_t height;
    int32_t width;
    int32_t channels;
    int32_t bytesPerPixel;
    const uint8_t* pixels;       // points into the original buffer
    std::size_t pixelByteCount;
};

inline int32_t readInt32LE(const uint8_t* p) {
    return static_cast<int32_t>(
        static_cast<uint32_t>(p[0])        |
        static_cast<uint32_t>(p[1]) <<  8  |
        static_cast<uint32_t>(p[2]) << 16  |
        static_cast<uint32_t>(p[3]) << 24);
}

AesthetixRawImage parseAesthetixRaw(const uint8_t* data, std::size_t size) {
    if (size < 16) throw std::runtime_error("Header too short");

    AesthetixRawImage img{};
    img.height        = readInt32LE(data + 0);
    img.width         = readInt32LE(data + 4);
    img.channels      = readInt32LE(data + 8);
    img.bytesPerPixel = readInt32LE(data + 12);

    const std::size_t expected =
        static_cast<std::size_t>(img.height) * img.width *
        img.channels * img.bytesPerPixel;
    if (size - 16 < expected) throw std::runtime_error("Pixel buffer too short");

    img.pixels = data + 16;
    img.pixelByteCount = expected;
    return img;
}

Building an OpenCV cv::Mat

OpenCV uses BGR internally, so it matches the Aesthetix colour layout without any cvtColor conversion.

#include <opencv2/core.hpp>

cv::Mat toCvMat(const AesthetixRawImage& img) {
    int cvType;
    if      (img.channels == 1 && img.bytesPerPixel == 1) cvType = CV_8UC1;
    else if (img.channels == 1 && img.bytesPerPixel == 4) cvType = CV_32FC1;
    else if (img.channels == 3 && img.bytesPerPixel == 1) cvType = CV_8UC3;  // BGR
    else throw std::runtime_error("Unsupported pixel format");

    // Wraps the bytes without copying; clone if the source buffer may be freed.
    return cv::Mat(img.height, img.width, cvType,
                   const_cast<uint8_t*>(img.pixels)).clone();
}

Normalising float images for display

// channels=1, bytesPerPixel=4 holds real float32 values (e.g. physical
// quantities) whose range is not bounded to [0,1]. Scale before rendering:
cv::Mat displayable;
cv::normalize(floatMat, displayable, 0, 255, cv::NORM_MINMAX, CV_8UC1);

Common pitfalls

• BGR is not RGB. Three-channel images use BGR ordering. Swap the channels (or rely on OpenCV's native BGR handling) before passing the data to libraries that expect RGB.
• Float images need scaling. bytesPerPixel = 4 images are real float32 data whose value range is not bounded to [0, 1]. Renderers must normalise the values.
• Read header values with memcpy or byte-shifting. A reinterpret_cast<int32_t*> is undefined behaviour on architectures with strict alignment when the buffer is not 4-byte aligned. The readInt32LE helper above sidesteps this and is also endian-portable.
• Compute the stride; don't hard-code it. Although the byte size today is Width * Channels * BytesPerPixel, computing the stride keeps your code robust if new pixel formats are introduced.
• Validate header values. Reject negative or implausibly large Height/Width values before multiplying them; a corrupt stream could otherwise overflow size_t.
• PNG/JPEG of post-processed images vs. raw originals. A handful of device images (surface, waviness, spot) are cropped server-side before being encoded as PNG or JPEG. The image/x-raw payload is the uncropped original. The shade images and most other images are not post-processed, so all three encodings cover identical pixel data.

See also

• Swagger UI Guide — explore the measurement-triggers endpoint interactively and try the different imageFormat values.
• Client Code Generation — generate a strongly typed REST client for C++ (cpp-restsdk) from the OpenAPI specification.