diff --git a/README.md b/README.md index 15387fb0..351c9f52 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ LightNVR provides a lightweight yet powerful solution for recording and managing #### đŸŽ¯ Smart Detection & Recording - **Detection Zones**: Visual polygon-based zone editor for targeted object detection - define multiple zones per camera with custom class filters and confidence thresholds -- **light-object-detect Integration**: Seamless integration with [light-object-detect](https://github.com/opensensor/light-object-detect) API for ONNX/TFLite-based object detection with zone filtering +- **light-object-detect Integration**: Seamless integration with [light-object-detect](https://github.com/opensensor/light-object-detect) API for ONNX/TFLite-based object detection with zone filtering; [DOODS2](https://github.com/snowzach/doods2) servers are supported through a selectable request format - **ONVIF Motion Events**: Automated recording triggered by ONVIF motion detection events - **Object Detection**: Optional SOD integration for motion and object detection (supports both RealNet and CNN models) @@ -295,6 +295,7 @@ Powerful object detection using modern ONNX and TFLite models with zone-aware fi - Enable **Detection Based Recording** - Set **API Detection URL** to `http://localhost:9001/api/v1/detect` - Choose detection backend: `onnx` (recommended), `tflite`, or `opencv` + - Running [DOODS2](https://github.com/snowzach/doods2) instead? Set **API Request Format** to DOODS2 in **Settings → Detection** and point the URL at its `/detect` endpoint (see [API Detection Settings](docs/CONFIGURATION.md#api-detection-settings)) - Configure **Detection Zones** to define areas of interest See [Zone Configuration Guide](docs/ZONE_CONFIGURATION.md) for detailed zone setup instructions. diff --git a/config/lightnvr.ini b/config/lightnvr.ini index fb87bab0..a374cb0d 100755 --- a/config/lightnvr.ini +++ b/config/lightnvr.ini @@ -94,6 +94,8 @@ path = /var/lib/lightnvr/data/models [api_detection] url = http://localhost:9001/api/v1/detect backend = onnx ; Detection backend: onnx (YOLOv8 - best accuracy), tflite, or opencv +format = light-object-detect ; Request format: light-object-detect (multipart upload) or doods2 (JSON body) +detector_name = default ; DOODS2 only: detector_name sent with each request confidence_threshold = 0.35 ; Lower threshold to catch distant vehicles filter_classes = car,motorcycle,truck,bus,bicycle ; Vehicle classes only diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 1c7508f2..db257884 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -79,10 +79,11 @@ max_streams = 32 path = /var/lib/lightnvr/data/models [api_detection] -url = http://localhost:9001/api/v1/detect -backend = onnx ; Detection backend: onnx (YOLOv8), tflite, or opencv -confidence_threshold = 0.35 -filter_classes = car,motorcycle,truck,bus,bicycle ; Comma-separated class filter +url = http://localhost:8000/api/v1/detect +format = light-object-detect ; Request format: light-object-detect or doods2 +backend = onnx ; light-object-detect backend: onnx (YOLOv8), tflite, or opencv +detector_name = default ; doods2 detector_name +detection_threshold = 50 [memory] buffer_size = 1024 ; Buffer size in KB @@ -364,16 +365,68 @@ path = /var/lib/lightnvr/data/models ```ini [api_detection] -url = http://localhost:9001/api/v1/detect -backend = onnx -confidence_threshold = 0.35 -filter_classes = car,motorcycle,truck,bus,bicycle +url = http://localhost:8000/api/v1/detect +format = light-object-detect ; or doods2 +backend = onnx ; light-object-detect only +detector_name = default ; doods2 only +detection_threshold = 50 ; Default confidence threshold (0-100%) ``` -- `url`: URL of the external detection API -- `backend`: Detection backend to use: `onnx` (YOLOv8 - best accuracy), `tflite`, or `opencv` -- `confidence_threshold`: Minimum confidence threshold for detections (0.0-1.0) -- `filter_classes`: Comma-separated list of object classes to detect (empty = all classes) +- `url`: URL of the external detection API. A stream's **Custom API Endpoint** + overrides it for that stream. Query parameters in either URL are passed + through to the server. +- `format`: Wire format used when posting snapshots: `light-object-detect` + (default) or `doods2`. Both are described below. +- `backend`: `light-object-detect` only. Inference backend requested from the + server: `onnx` (YOLOv8 - best accuracy), `tflite`, or `opencv`. +- `detector_name`: `doods2` only. The `detector_name` sent with each request; + it must match one of the detectors the server lists at `GET /detectors`. +- `detection_threshold`: Default confidence threshold (0-100%) for streams that + do not set their own. + +A stream whose only detection engine is an `api` engine may override `format`, +`backend` and `detector_name` in that engine's `config` object via +`PUT /api/streams/{name}/detection-engines`; see +[Detection Engines](DETECTION_ENGINES.md). + +#### Wire formats + +LightNVR grabs one JPEG per detection interval and posts it to the API. It +never asks the detector to open the camera stream itself, so streaming request +modes offered by some servers are not used. + +**light-object-detect** (default) + +Request: `POST ?backend=&confidence_threshold=<0-1>&return_image=false` +as `multipart/form-data` with a single `file` part containing the JPEG. + +Response: + +```json +{"detections": [ + {"label": "person", "confidence": 0.91, + "x_min": 0.10, "y_min": 0.20, "x_max": 0.40, "y_max": 0.80} +]} +``` + +Coordinates are normalized 0-1 and `confidence` is 0-1. The box may instead be +nested under a `bounding_box` object. Optional `track_id` (number) and +`zone_id` (string) are stored when present. Any server that implements this +contract works with the default format. + +**doods2** + +Request: `POST ` with `Content-Type: application/json`: + +```json +{"id": "", "detector_name": "", + "detect": {"*": }, "data": ""} +``` + +Response: the standard [DOODS2](https://github.com/snowzach/doods2) detect +response. `top`/`left`/`bottom`/`right` are normalized 0-1 and `confidence` is +0-100; LightNVR converts both to its internal 0-1 scale. A non-empty `error` +field fails the request. ### Memory Optimization diff --git a/docs/DETECTION_ENGINES.md b/docs/DETECTION_ENGINES.md index 79399da2..7811b478 100644 --- a/docs/DETECTION_ENGINES.md +++ b/docs/DETECTION_ENGINES.md @@ -53,6 +53,27 @@ rows including the compatibility engine. Relative object model paths resolve under the configured models directory. `config` must be a bounded JSON object and must not contain credentials. +For `api` engines, `model_path` is the detection endpoint URL and `config` may +carry request options that override the global `[api_detection]` settings: +`format` (`light-object-detect` or `doods2`), `backend` (light-object-detect) +and `detector_name` (doods2). See the wire formats in +[CONFIGURATION.md](CONFIGURATION.md#wire-formats). + +```json +{ + "key": "doods", + "type": "api", + "model_path": "http://doods:8080/detect", + "enabled": true, + "threshold": 0.4, + "interval_seconds": 2, + "config": {"format": "doods2", "detector_name": "tensorflow"} +} +``` + +The override takes effect when this is the stream's only engine (leave the +stream's legacy detection model empty); see the runtime boundaries below. + ## Runtime boundaries - Motion and local object engines share decoded frames and run at their own diff --git a/include/core/config.h b/include/core/config.h index d95350b5..8790a64a 100644 --- a/include/core/config.h +++ b/include/core/config.h @@ -6,6 +6,19 @@ #include "telemetry/system_health_policy.h" +// Wire formats supported by the external detection API client (api_detection.c). +#define API_DETECTION_FORMAT_MAX 32 +#define API_DETECTION_BACKEND_MAX 32 +#define API_DETECTION_DETECTOR_NAME_MAX 64 +#define API_DETECTION_FORMAT_NAME_LIGHT_OBJECT_DETECT "light-object-detect" +#define API_DETECTION_FORMAT_NAME_DOODS2 "doods2" + +typedef enum { + API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT = 0, // multipart "file" upload + query params + API_DETECTION_FORMAT_DOODS2 = 1, // JSON body with base64 "data" +} api_detection_format_t; + + // Maximum length for path strings #define MAX_PATH_LENGTH 512 // Maximum length for stream names @@ -227,6 +240,8 @@ typedef struct { // API detection settings char api_detection_url[MAX_URL_LENGTH]; // URL for the detection API char api_detection_backend[32]; // Backend to use: onnx, tflite, opencv (default: onnx) + char api_detection_format[API_DETECTION_FORMAT_MAX]; // Wire format: light-object-detect (default) or doods2 + char api_detection_detector_name[API_DETECTION_DETECTOR_NAME_MAX]; // DOODS2 detector_name (default: "default") // Global detection defaults (used when per-stream settings are not specified) int default_detection_threshold; // Default confidence threshold for detection (0-100) @@ -473,4 +488,21 @@ static inline int configured_stream_slots(void) { return slots; } +/** + * Canonical name for an API detection wire format. + */ +const char *api_detection_format_name(api_detection_format_t format); + +/** + * Parse a format name (case-insensitive; accepts "light-object-detect"/"lod" + * and "doods2"/"doods"). Returns false for unknown names, leaving *out untouched. + */ +bool api_detection_format_parse(const char *name, api_detection_format_t *out); + +/** + * Store the canonical name of a parsed format in config->api_detection_format. + * Returns false (and leaves the config untouched) for unknown names. + */ +bool config_set_api_detection_format(config_t *config, const char *name); + #endif /* LIGHTNVR_CONFIG_H */ diff --git a/include/utils/base64.h b/include/utils/base64.h new file mode 100644 index 00000000..6c880e65 --- /dev/null +++ b/include/utils/base64.h @@ -0,0 +1,27 @@ +#ifndef LIGHTNVR_UTILS_BASE64_H +#define LIGHTNVR_UTILS_BASE64_H + +#include + +/** + * Bytes required to hold the standard (RFC 4648, padded) base64 encoding of + * input_len bytes, including the terminating NUL. Returns 0 on overflow. + */ +size_t base64_encoded_size(size_t input_len); + +/** + * Encode input into dst as NUL-terminated base64. + * + * @param out_len Optional; receives the encoded length (without the NUL). + * @return 0 on success, -1 when dst is too small or the arguments are invalid. + */ +int base64_encode(const unsigned char *input, size_t input_len, + char *dst, size_t dst_size, size_t *out_len); + +/** + * Convenience wrapper that returns a malloc'd NUL-terminated encoding. + * The caller frees the result. Returns NULL on failure. + */ +char *base64_encode_alloc(const unsigned char *input, size_t input_len); + +#endif /* LIGHTNVR_UTILS_BASE64_H */ diff --git a/include/video/api_detection.h b/include/video/api_detection.h index 959a56a7..d7c0a8ca 100644 --- a/include/video/api_detection.h +++ b/include/video/api_detection.h @@ -2,80 +2,112 @@ #define LIGHTNVR_API_DETECTION_H #include +#include #include #include + +#include "core/config.h" #include "video/detection_result.h" // Model type for API-based detection #define MODEL_TYPE_API "api" /** - * Initialize the API detection system - * - * @return 0 on success, non-zero on failure + * Per-request options for the HTTP detection API. + * + * LightNVR speaks two wire formats: + * + * - light-object-detect (default): multipart/form-data upload of the JPEG in a + * "file" part, with backend/confidence_threshold/return_image query + * parameters appended to the URL. The reply is {"detections":[{label, + * confidence 0-1, x_min/y_min/x_max/y_max normalized 0-1 (flat or under + * "bounding_box")}]}. + * + * - doods2: JSON body {"id","detector_name","detect":{"*":percent},"data": + * base64 JPEG} posted verbatim to the URL. The reply is {"detections":[{ + * top/left/bottom/right normalized 0-1, label, confidence 0-100}],"error"}. + * + * The global defaults come from [api_detection] in lightnvr.ini and may be + * overridden per stream through the detection engine "config" JSON object + * ({"format","backend","detector_name"}). */ -int init_api_detection_system(void); +typedef struct { + api_detection_format_t format; + char backend[API_DETECTION_BACKEND_MAX]; // light-object-detect: inference backend query param + char detector_name[API_DETECTION_DETECTOR_NAME_MAX]; // doods2: "detector_name" request field +} api_detection_options_t; + +/** Populate options from the global configuration. */ +void api_detection_options_from_config(api_detection_options_t *options); /** - * Shutdown the API detection system + * Overlay per-engine overrides from a detection engine config JSON object. + * Unknown keys are ignored. Returns 0 on success (including an empty or + * absent config) and -1 when the JSON is malformed or names an unknown + * format; options are left in a consistent state either way. */ -void shutdown_api_detection_system(void); +int api_detection_options_apply_json(api_detection_options_t *options, + const char *config_json); /** - * Detect objects using the API - * - * Uses the provided decoded frame when available. If no decoded frame is - * supplied and a stream name is available, the implementation may fetch a - * JPEG snapshot from go2rtc instead. - * - * @param api_url The URL of the detection API - * @param frame_data The frame data to detect objects in - * @param width The width of the frame - * @param height The height of the frame - * @param channels The number of channels in the frame - * @param result Pointer to a detection_result_t structure to store the results - * @param stream_name The name of the stream (for database storage) - * @param threshold Confidence threshold for detection (0.0-1.0, use negative for default) - * @param recording_id Recording ID to link detections to (0 for no link) - * @param frame_timestamp Wall-clock time when the frame entered the detection - * pipeline. Used as the DB/MQTT event timestamp so it - * reflects frame time rather than inference completion. - * Pass 0 to fall back to time(NULL) inside the DB layer. - * @return 0 on success, non-zero on failure + * Build the request URL for the selected format. light-object-detect appends + * its query parameters; doods2 uses the base URL verbatim. NULL options mean + * the global configuration. Returns 0 on success, -1 on error. + */ +int api_detection_build_request_url(char *buffer, size_t buffer_size, + const char *base_url, float threshold, + const api_detection_options_t *options); + +/** + * Build the DOODS2 JSON request body for a JPEG snapshot. request_id is sent + * back by DOODS2 in the response "id" field (stream name when available). + * Returns a malloc'd string the caller frees, or NULL on failure. + */ +char *api_detection_build_doods2_body(const unsigned char *jpeg_data, size_t jpeg_size, + float threshold, const char *request_id, + const api_detection_options_t *options); + +/** + * Parse a detection API response into result (reset first). Both box shapes + * are accepted regardless of format; the confidence scale follows the format + * (0-1 for light-object-detect, 0-100 for doods2). Returns 0 on success and + * -1 when the document is not a detection response or reports an error. */ +int api_detection_parse_response(const char *json, + const api_detection_options_t *options, + detection_result_t *result); + +int init_api_detection_system(void); + +void shutdown_api_detection_system(void); + int detect_objects_api(const char *api_url, const unsigned char *frame_data, int width, int height, int channels, detection_result_t *result, const char *stream_name, float threshold, uint64_t recording_id, time_t frame_timestamp); -/** - * @brief Determine whether API detection should fetch a go2rtc snapshot. - * - * This prevents fallback callers from re-entering the go2rtc snapshot path - * after they already decoded a local frame for the same detection attempt. - */ +/** Same as detect_objects_api with explicit request options (NULL = global config). */ +int detect_objects_api_with_options(const char *api_url, const unsigned char *frame_data, + int width, int height, int channels, + detection_result_t *result, const char *stream_name, + float threshold, uint64_t recording_id, + time_t frame_timestamp, + const api_detection_options_t *options); + bool api_detection_should_use_go2rtc_snapshot(const unsigned char *frame_data, int width, int height, int channels, const char *stream_name); -/** - * Detect objects using the API with go2rtc snapshot only (no frame data required) - * - * This function fetches a snapshot directly from go2rtc and sends it to the detection API. - * It does NOT require decoded frame data, which saves significant memory by avoiding - * the need to decode video segments. - * - * @param api_url The URL of the detection API - * @param stream_name The name of the stream (required for go2rtc snapshot) - * @param result Pointer to a detection_result_t structure to store the results - * @param threshold Confidence threshold for detection (0.0-1.0, use negative for default) - * @param recording_id Recording ID to link detections to (0 for no link) - * @return 0 on success, -1 on general failure, -2 if go2rtc snapshot failed (caller should fall back) - */ int detect_objects_api_snapshot(const char *api_url, const char *stream_name, detection_result_t *result, float threshold, uint64_t recording_id, time_t frame_timestamp); +/** Same as detect_objects_api_snapshot with explicit request options (NULL = global config). */ +int detect_objects_api_snapshot_with_options(const char *api_url, const char *stream_name, + detection_result_t *result, float threshold, + uint64_t recording_id, time_t frame_timestamp, + const api_detection_options_t *options); + #endif /* LIGHTNVR_API_DETECTION_H */ diff --git a/src/core/config.c b/src/core/config.c index 57130d6e..32857fbb 100644 --- a/src/core/config.c +++ b/src/core/config.c @@ -379,8 +379,12 @@ void load_default_config(config_t *config) { safe_strcpy(config->models_path, "/var/lib/lightnvr/models", MAX_PATH_LENGTH, 0); // API detection settings - safe_strcpy(config->api_detection_url, "http://localhost:8000/detect", MAX_URL_LENGTH, 0); + safe_strcpy(config->api_detection_url, "http://localhost:8000/api/v1/detect", MAX_URL_LENGTH, 0); safe_strcpy(config->api_detection_backend, "onnx", 32, 0); // Default to ONNX backend + safe_strcpy(config->api_detection_format, API_DETECTION_FORMAT_NAME_LIGHT_OBJECT_DETECT, + sizeof(config->api_detection_format), 0); + safe_strcpy(config->api_detection_detector_name, "default", + sizeof(config->api_detection_detector_name), 0); // Global detection defaults config->default_detection_threshold = 50; // 50% confidence threshold @@ -853,6 +857,15 @@ static int config_ini_handler(void* user, const char* section, const char* name, safe_strcpy(config->api_detection_url, value, MAX_URL_LENGTH, 0); } else if (strcmp(name, "backend") == 0) { safe_strcpy(config->api_detection_backend, value, sizeof(config->api_detection_backend), 0); + } else if (strcmp(name, "format") == 0) { + if (!config_set_api_detection_format(config, value)) { + log_warn("Unknown [api_detection] format '%s'; expected %s or %s, keeping %s", + value, API_DETECTION_FORMAT_NAME_LIGHT_OBJECT_DETECT, + API_DETECTION_FORMAT_NAME_DOODS2, config->api_detection_format); + } + } else if (strcmp(name, "detector_name") == 0) { + safe_strcpy(config->api_detection_detector_name, value, + sizeof(config->api_detection_detector_name), 0); } else if (strcmp(name, "detection_threshold") == 0) { config->default_detection_threshold = safe_atoi(value, 0); // Clamp to valid range @@ -1826,6 +1839,10 @@ int save_config(const config_t *config, const char *path) { fprintf(file, "[api_detection]\n"); fprintf(file, "url = %s\n", config->api_detection_url); fprintf(file, "backend = %s\n", config->api_detection_backend); + fprintf(file, "format = %s ; Request format: light-object-detect (multipart upload) or doods2 (JSON body)\n", + config->api_detection_format); + fprintf(file, "detector_name = %s ; DOODS2 only: detector_name sent with each request\n", + config->api_detection_detector_name); fprintf(file, "detection_threshold = %d ; Default confidence threshold (0-100%%)\n", config->default_detection_threshold); fprintf(file, "pre_detection_buffer = %d\n", config->default_pre_detection_buffer); fprintf(file, "post_detection_buffer = %d\n", config->default_post_detection_buffer); @@ -2089,3 +2106,62 @@ void print_config(const config_t *config) { } } } + +/* ------------------------------------------------------------------------- */ +/* API detection wire format helpers */ +/* ------------------------------------------------------------------------- */ + +const char *api_detection_format_name(api_detection_format_t format) { + switch (format) { + case API_DETECTION_FORMAT_DOODS2: + return API_DETECTION_FORMAT_NAME_DOODS2; + case API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT: + default: + return API_DETECTION_FORMAT_NAME_LIGHT_OBJECT_DETECT; + } +} + +bool api_detection_format_parse(const char *name, api_detection_format_t *out) { + if (!name || !out) { + return false; + } + + // Tolerate surrounding whitespace from hand-edited ini files. + while (*name == ' ' || *name == '\t') { + name++; + } + size_t len = strlen(name); + while (len > 0 && (name[len - 1] == ' ' || name[len - 1] == '\t')) { + len--; + } + if (len == 0 || len >= API_DETECTION_FORMAT_MAX) { + return false; + } + + char trimmed[API_DETECTION_FORMAT_MAX]; + memcpy(trimmed, name, len); + trimmed[len] = '\0'; + + if (strcasecmp(trimmed, API_DETECTION_FORMAT_NAME_LIGHT_OBJECT_DETECT) == 0 || + strcasecmp(trimmed, "light_object_detect") == 0 || + strcasecmp(trimmed, "lod") == 0) { + *out = API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT; + return true; + } + if (strcasecmp(trimmed, API_DETECTION_FORMAT_NAME_DOODS2) == 0 || + strcasecmp(trimmed, "doods") == 0) { + *out = API_DETECTION_FORMAT_DOODS2; + return true; + } + return false; +} + +bool config_set_api_detection_format(config_t *config, const char *name) { + api_detection_format_t format; + if (!config || !api_detection_format_parse(name, &format)) { + return false; + } + safe_strcpy(config->api_detection_format, api_detection_format_name(format), + sizeof(config->api_detection_format), 0); + return true; +} diff --git a/src/utils/base64.c b/src/utils/base64.c new file mode 100644 index 00000000..15443383 --- /dev/null +++ b/src/utils/base64.c @@ -0,0 +1,80 @@ +#include +#include + +#include "utils/base64.h" + +static const char BASE64_ALPHABET[] = + "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + +size_t base64_encoded_size(size_t input_len) { + /* Every 3 input bytes become 4 output characters; the last group is padded. */ + if (input_len > (SIZE_MAX - 1) / 4 * 3 - 2) { + return 0; + } + return ((input_len + 2) / 3) * 4 + 1; +} + +int base64_encode(const unsigned char *input, size_t input_len, + char *dst, size_t dst_size, size_t *out_len) { + if (out_len) { + *out_len = 0; + } + if (!dst || dst_size == 0 || (!input && input_len > 0)) { + return -1; + } + + size_t needed = base64_encoded_size(input_len); + if (needed == 0 || dst_size < needed) { + return -1; + } + + size_t written = 0; + size_t i = 0; + while (i + 2 < input_len) { + uint32_t triple = ((uint32_t)input[i] << 16) | + ((uint32_t)input[i + 1] << 8) | + (uint32_t)input[i + 2]; + dst[written++] = BASE64_ALPHABET[(triple >> 18) & 0x3F]; + dst[written++] = BASE64_ALPHABET[(triple >> 12) & 0x3F]; + dst[written++] = BASE64_ALPHABET[(triple >> 6) & 0x3F]; + dst[written++] = BASE64_ALPHABET[triple & 0x3F]; + i += 3; + } + + size_t remaining = input_len - i; + if (remaining == 1) { + uint32_t triple = (uint32_t)input[i] << 16; + dst[written++] = BASE64_ALPHABET[(triple >> 18) & 0x3F]; + dst[written++] = BASE64_ALPHABET[(triple >> 12) & 0x3F]; + dst[written++] = '='; + dst[written++] = '='; + } else if (remaining == 2) { + uint32_t triple = ((uint32_t)input[i] << 16) | ((uint32_t)input[i + 1] << 8); + dst[written++] = BASE64_ALPHABET[(triple >> 18) & 0x3F]; + dst[written++] = BASE64_ALPHABET[(triple >> 12) & 0x3F]; + dst[written++] = BASE64_ALPHABET[(triple >> 6) & 0x3F]; + dst[written++] = '='; + } + + dst[written] = '\0'; + if (out_len) { + *out_len = written; + } + return 0; +} + +char *base64_encode_alloc(const unsigned char *input, size_t input_len) { + size_t needed = base64_encoded_size(input_len); + if (needed == 0) { + return NULL; + } + char *encoded = malloc(needed); + if (!encoded) { + return NULL; + } + if (base64_encode(input, input_len, encoded, needed, NULL) != 0) { + free(encoded); + return NULL; + } + return encoded; +} diff --git a/src/video/api_detection.c b/src/video/api_detection.c index e9bd0685..2effaa92 100644 --- a/src/video/api_detection.c +++ b/src/video/api_detection.c @@ -1,6 +1,7 @@ #include #include #include +#include #include #include #include @@ -15,6 +16,7 @@ #include "core/curl_init.h" #include "core/shutdown_coordinator.h" #include "core/event_producers.h" +#include "utils/base64.h" #include "utils/strings.h" #include "video/api_detection.h" #include "video/detection_result.h" @@ -42,10 +44,20 @@ static pthread_mutex_t curl_mutex = PTHREAD_MUTEX_INITIALIZER; // Initial buffer size (in bytes) for CURL responses to reduce realloc churn. #define API_DETECTION_INITIAL_RESPONSE_BUFFER_SIZE 1024 +// Maximum length of the fully built request URL. +#define API_DETECTION_REQUEST_URL_MAX 1024 + +// DOODS2 request body up to (and including) the opening quote of the base64 "data" value. +#define DOODS2_BODY_PREFIX_FORMAT "{\"id\":%s,\"detector_name\":%s,\"detect\":{\"*\":%.2f},\"data\":\"" + // ASCII printable character range used when sanitizing response previews. #define ASCII_PRINTABLE_MIN 32 #define ASCII_PRINTABLE_MAX 126 +// Returned by the snapshot entry point when go2rtc cannot supply a frame; the +// caller falls back to the decode path. Mirrors DETECT_SNAPSHOT_UNAVAILABLE. +#define API_DETECTION_SNAPSHOT_UNAVAILABLE -2 + // Structure to hold memory for curl response typedef struct { char *memory; @@ -185,38 +197,341 @@ static bool validate_api_detection_base_url(const char *base_url, const char *co return true; } -// Helper to build the API detection URL with common query parameters. -// Returns 0 on success, -1 on error (e.g., buffer too small or invalid args). -static int build_api_detection_url(char *buffer, - size_t buffer_size, - const char *base_url, - const char *backend, - float threshold, - bool return_image_flag) { +// Resolve the special "api-detection" model path to the globally configured URL. +static const char *resolve_api_url(const char *api_url, const char *context) { + if (api_url && strcmp(api_url, "api-detection") == 0) { + log_info("%s: Using API URL from config: %s", context, + g_config.api_detection_url[0] ? g_config.api_detection_url : "NULL"); + return g_config.api_detection_url; + } + return api_url; +} + +/* ------------------------------------------------------------------------- */ +/* Request options */ +/* ------------------------------------------------------------------------- */ + +void api_detection_options_from_config(api_detection_options_t *options) { + if (!options) { + return; + } + memset(options, 0, sizeof(*options)); + + api_detection_format_t format = API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT; + if (g_config.api_detection_format[0] != '\0' && + !api_detection_format_parse(g_config.api_detection_format, &format)) { + log_warn("API Detection: Unknown api_detection format '%s', using %s", + g_config.api_detection_format, + api_detection_format_name(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT)); + format = API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT; + } + options->format = format; + safe_strcpy(options->backend, g_config.api_detection_backend, sizeof(options->backend), 0); + safe_strcpy(options->detector_name, g_config.api_detection_detector_name, + sizeof(options->detector_name), 0); +} + +int api_detection_options_apply_json(api_detection_options_t *options, + const char *config_json) { + if (!options) { + return -1; + } + if (!config_json || config_json[0] == '\0') { + return 0; + } + + cJSON *root = cJSON_Parse(config_json); + if (!root) { + log_warn("API Detection: Engine config is not valid JSON"); + return -1; + } + + int rc = 0; + if (!cJSON_IsObject(root)) { + log_warn("API Detection: Engine config must be a JSON object"); + rc = -1; + } else { + const cJSON *format = cJSON_GetObjectItemCaseSensitive(root, "format"); + if (format && !cJSON_IsNull(format)) { + api_detection_format_t parsed; + if (cJSON_IsString(format) && api_detection_format_parse(format->valuestring, &parsed)) { + options->format = parsed; + } else { + log_warn("API Detection: Engine config names an unknown format%s%s", + cJSON_IsString(format) ? ": " : "", + cJSON_IsString(format) ? format->valuestring : ""); + rc = -1; + } + } + + const cJSON *backend = cJSON_GetObjectItemCaseSensitive(root, "backend"); + if (cJSON_IsString(backend) && backend->valuestring[0] != '\0') { + safe_strcpy(options->backend, backend->valuestring, sizeof(options->backend), 0); + } + + const cJSON *detector = cJSON_GetObjectItemCaseSensitive(root, "detector_name"); + if (cJSON_IsString(detector) && detector->valuestring[0] != '\0') { + safe_strcpy(options->detector_name, detector->valuestring, + sizeof(options->detector_name), 0); + } + } + + cJSON_Delete(root); + return rc; +} + +static const api_detection_options_t *resolve_options(const api_detection_options_t *options, + api_detection_options_t *storage) { + if (options) { + return options; + } + api_detection_options_from_config(storage); + return storage; +} + +static const char *effective_detector_name(const api_detection_options_t *options) { + return (options->detector_name[0] != '\0') ? options->detector_name : "default"; +} + +/* ------------------------------------------------------------------------- */ +/* Request construction */ +/* ------------------------------------------------------------------------- */ + +int api_detection_build_request_url(char *buffer, size_t buffer_size, + const char *base_url, float threshold, + const api_detection_options_t *options) { if (buffer == NULL || buffer_size == 0 || base_url == NULL || !is_safe_base_url(base_url)) { return -1; } - const char *backend_param = sanitize_backend(backend); - float actual_threshold = normalize_api_detection_threshold(threshold); - const char *return_image_value = return_image_flag ? "true" : "false"; - char separator = (strchr(base_url, '?') != NULL) ? '&' : '?'; + api_detection_options_t storage; + options = resolve_options(options, &storage); + + int url_len; + if (options->format == API_DETECTION_FORMAT_DOODS2) { + // DOODS2 carries every parameter in the JSON body; the URL is used verbatim. + url_len = snprintf(buffer, buffer_size, "%s", base_url); + } else { + const char *backend_param = sanitize_backend(options->backend); + float actual_threshold = normalize_api_detection_threshold(threshold); + char separator = (strchr(base_url, '?') != NULL) ? '&' : '?'; + + url_len = snprintf(buffer, buffer_size, + "%s%cbackend=%s&confidence_threshold=%.2f&return_image=false", + base_url, separator, backend_param, actual_threshold); + } - int url_len = snprintf(buffer, - buffer_size, - "%s%cbackend=%s&confidence_threshold=%.2f&return_image=%s", - base_url, - separator, - backend_param, - actual_threshold, - return_image_value); if (url_len < 0 || (size_t)url_len >= buffer_size) { return -1; } + return 0; +} + +// JSON-quote a string (including the surrounding double quotes). Caller frees. +static char *json_quote(const char *text) { + cJSON *item = cJSON_CreateString(text ? text : ""); + if (!item) { + return NULL; + } + char *quoted = cJSON_PrintUnformatted(item); + cJSON_Delete(item); + return quoted; +} + +char *api_detection_build_doods2_body(const unsigned char *jpeg_data, size_t jpeg_size, + float threshold, const char *request_id, + const api_detection_options_t *options) { + if (!jpeg_data || jpeg_size == 0) { + return NULL; + } + + api_detection_options_t storage; + options = resolve_options(options, &storage); + + char *id_json = json_quote((request_id && request_id[0]) ? request_id : "lightnvr"); + char *detector_json = json_quote(effective_detector_name(options)); + char *encoded = base64_encode_alloc(jpeg_data, jpeg_size); + char *body = NULL; + + if (id_json && detector_json && encoded) { + // DOODS2 filters on a 0-100 confidence scale; "*" applies to every label. + double percent = (double)normalize_api_detection_threshold(threshold) * 100.0; + int prefix_len = snprintf(NULL, 0, DOODS2_BODY_PREFIX_FORMAT, id_json, detector_json, percent); + size_t encoded_len = strlen(encoded); + if (prefix_len > 0) { + size_t total = (size_t)prefix_len + encoded_len + 3; // closing quote, brace, NUL + body = malloc(total); + if (body) { + int written = snprintf(body, total, DOODS2_BODY_PREFIX_FORMAT, id_json, detector_json, percent); + if (written == prefix_len) { + memcpy(body + written, encoded, encoded_len); + body[written + encoded_len] = '"'; + body[written + encoded_len + 1] = '}'; + body[written + encoded_len + 2] = '\0'; + } else { + free(body); + body = NULL; + } + } + } + } + + if (!body) { + log_error("API Detection: Failed to build DOODS2 request body (%zu byte JPEG)", jpeg_size); + } + + free(id_json); + free(detector_json); + free(encoded); + return body; +} + +/* ------------------------------------------------------------------------- */ +/* Response parsing */ +/* ------------------------------------------------------------------------- */ + +static bool number_item(const cJSON *object, const char *key, double *out) { + const cJSON *item = object ? cJSON_GetObjectItem(object, key) : NULL; + if (!item || !cJSON_IsNumber(item)) { + return false; + } + *out = item->valuedouble; + return true; +} + +// Extract a normalized box from either the light-object-detect shape +// (x_min/y_min/x_max/y_max, flat or nested under "bounding_box") or the +// DOODS2 shape (top/left/bottom/right). +static bool extract_box(const cJSON *detection, double *x_min, double *y_min, + double *x_max, double *y_max) { + const cJSON *nested = cJSON_GetObjectItem(detection, "bounding_box"); + const cJSON *source = (nested && cJSON_IsObject(nested)) ? nested : detection; + + if (number_item(source, "x_min", x_min) && number_item(source, "y_min", y_min) && + number_item(source, "x_max", x_max) && number_item(source, "y_max", y_max)) { + return true; + } + + return number_item(detection, "left", x_min) && number_item(detection, "top", y_min) && + number_item(detection, "right", x_max) && number_item(detection, "bottom", y_max); +} + +static void log_json_item(const char *message, const cJSON *item, bool as_error) { + char *json_str = cJSON_Print(item); + if (!json_str) { + return; + } + if (as_error) { + log_error("%s: %s", message, json_str); + } else { + log_warn("%s: %s", message, json_str); + } + free(json_str); +} + +int api_detection_parse_response(const char *json, + const api_detection_options_t *options, + detection_result_t *result) { + if (!json || !result) { + return -1; + } + memset(result, 0, sizeof(*result)); + + api_detection_options_t storage; + options = resolve_options(options, &storage); + const bool doods2 = (options->format == API_DETECTION_FORMAT_DOODS2); + + cJSON *root = cJSON_Parse(json); + if (!root) { + const char *error_ptr = cJSON_GetErrorPtr(); + log_error("API Detection: Failed to parse JSON response: %s", error_ptr ? error_ptr : "Unknown error"); + return -1; + } + + if (doods2) { + const cJSON *error = cJSON_GetObjectItem(root, "error"); + if (error && cJSON_IsString(error) && error->valuestring[0] != '\0') { + log_error("API Detection: DOODS2 reported an error: %s", error->valuestring); + cJSON_Delete(root); + return -1; + } + } + + const cJSON *detections = cJSON_GetObjectItem(root, "detections"); + if (detections && cJSON_IsNull(detections)) { + // An explicit null is "nothing detected", not a malformed reply. + cJSON_Delete(root); + return 0; + } + if (!detections || !cJSON_IsArray(detections)) { + log_error("API Detection: Invalid JSON response: missing or invalid 'detections' array"); + log_json_item("API Detection: Full JSON response", root, true); + cJSON_Delete(root); + return -1; + } + + int array_size = cJSON_GetArraySize(detections); + for (int i = 0; i < array_size; i++) { + if (result->count >= MAX_DETECTIONS) { + log_warn("API Detection: Maximum number of detections reached (%d)", MAX_DETECTIONS); + break; + } + + const cJSON *detection = cJSON_GetArrayItem(detections, i); + if (!detection || !cJSON_IsObject(detection)) { + continue; + } + + const cJSON *label = cJSON_GetObjectItem(detection, "label"); + double confidence = 0.0; + double x_min = 0.0, y_min = 0.0, x_max = 0.0, y_max = 0.0; + + if (!label || !cJSON_IsString(label) || + !number_item(detection, "confidence", &confidence) || + !extract_box(detection, &x_min, &y_min, &x_max, &y_max)) { + log_warn("API Detection: Invalid detection data in JSON response"); + log_json_item("API Detection: Detection JSON", detection, false); + continue; + } + + if (doods2) { + confidence /= 100.0; + } + if (confidence < 0.0) { + confidence = 0.0; + } else if (confidence > 1.0) { + confidence = 1.0; + } + + detection_t *entry = &result->detections[result->count]; + safe_strcpy(entry->label, label->valuestring, MAX_LABEL_LENGTH, 0); + entry->confidence = (float)confidence; + entry->x = (float)x_min; + entry->y = (float)y_min; + entry->width = (float)(x_max - x_min); + entry->height = (float)(y_max - y_min); + + const cJSON *track_id = cJSON_GetObjectItem(detection, "track_id"); + entry->track_id = (track_id && cJSON_IsNumber(track_id)) ? (int)track_id->valuedouble : -1; + + const cJSON *zone_id = cJSON_GetObjectItem(detection, "zone_id"); + if (zone_id && cJSON_IsString(zone_id)) { + safe_strcpy(entry->zone_id, zone_id->valuestring, MAX_ZONE_ID_LENGTH, 0); + } else { + entry->zone_id[0] = '\0'; + } + + result->count++; + } + cJSON_Delete(root); return 0; } +/* ------------------------------------------------------------------------- */ +/* HTTP transport */ +/* ------------------------------------------------------------------------- */ + // Callback function for curl to write data static size_t write_memory_callback(void *contents, size_t size, size_t nmemb, void *userp) { if (nmemb != 0 && size > (SIZE_MAX / nmemb)) { @@ -302,6 +617,213 @@ static void log_tls_error_details(const char *context, CURL *curl, CURLcode res, context); } +// Copy a printable prefix of the response body for logging. +static void make_response_preview(const memory_struct_t *chunk, + char *preview, size_t preview_size) { + size_t copy_len = 0; + if (chunk->memory && preview_size > 0) { + copy_len = chunk->size < (preview_size - 1) ? chunk->size : (preview_size - 1); + memcpy(preview, chunk->memory, copy_len); + } + if (preview_size > 0) { + preview[copy_len] = '\0'; + } + for (size_t i = 0; i < copy_len; i++) { + if (preview[i] < ASCII_PRINTABLE_MIN || preview[i] > ASCII_PRINTABLE_MAX) { + preview[i] = '.'; + } + } +} + +// Build the light-object-detect multipart body: one "file" part holding the JPEG. +// curl_mime_data copies the bytes, so the caller may free jpeg_data afterwards. +static curl_mime *build_multipart_body(CURL *curl, const unsigned char *jpeg_data, + size_t jpeg_size, const char *context) { + curl_mime *mime = curl_mime_init(curl); + if (!mime) { + log_error("%s: Failed to create mime structure", context); + return NULL; + } + + curl_mimepart *part = curl_mime_addpart(mime); + if (!part) { + log_error("%s: Failed to add mime part", context); + curl_mime_free(mime); + return NULL; + } + + CURLcode rc; + if ((rc = curl_mime_name(part, "file")) != CURLE_OK || + (rc = curl_mime_data(part, (const char *)jpeg_data, jpeg_size)) != CURLE_OK || + (rc = curl_mime_filename(part, "snapshot.jpg")) != CURLE_OK || + (rc = curl_mime_type(part, "image/jpeg")) != CURLE_OK) { + log_error("%s: Failed to build multipart body: %s", context, curl_easy_strerror(rc)); + curl_mime_free(mime); + return NULL; + } + + return mime; +} + +/** + * Send one JPEG to the detection API using the selected wire format and parse + * the reply into result. Does not touch the database. + * + * Returns 0 on success, -1 on any transport, HTTP, or parse failure. + */ +static int perform_detection_request(const char *context, const char *base_url, + const unsigned char *jpeg_data, size_t jpeg_size, + float threshold, const char *request_id, + const api_detection_options_t *options, + detection_result_t *result) { + CURL *curl = NULL; + curl_mime *mime = NULL; + struct curl_slist *headers = NULL; + char *body = NULL; + memory_struct_t chunk = {0}; + char request_url[API_DETECTION_REQUEST_URL_MAX]; + char preview[API_DETECTION_RESPONSE_PREVIEW_LEN]; + int ret = -1; + + if (api_detection_build_request_url(request_url, sizeof(request_url), + base_url, threshold, options) != 0) { + log_error("%s: Failed to construct request URL for %s", context, base_url); + return -1; + } + + // Use a per-call curl handle so detection requests can run concurrently. + curl = curl_easy_init(); + if (curl == NULL) { + log_error("%s: Failed to initialize CURL handle", context); + return -1; + } + + if (options->format == API_DETECTION_FORMAT_DOODS2) { + body = api_detection_build_doods2_body(jpeg_data, jpeg_size, threshold, request_id, options); + if (!body) { + goto cleanup; + } + headers = curl_slist_append(headers, "Content-Type: application/json"); + curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body); + curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t)strlen(body)); + log_info("%s: Sending DOODS2 request to %s (detector: %s, threshold: %.2f, %zu byte JPEG)", + context, request_url, effective_detector_name(options), + normalize_api_detection_threshold(threshold), jpeg_size); + } else { + mime = build_multipart_body(curl, jpeg_data, jpeg_size, context); + if (!mime) { + goto cleanup; + } + curl_easy_setopt(curl, CURLOPT_MIMEPOST, mime); + log_info("%s: Sending request to %s (backend: %s, threshold: %.2f, %zu byte JPEG)", + context, request_url, sanitize_backend(options->backend), + normalize_api_detection_threshold(threshold), jpeg_size); + } + + headers = curl_slist_append(headers, "accept: application/json"); + curl_easy_setopt(curl, CURLOPT_URL, request_url); + curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers); + + chunk.memory = malloc(API_DETECTION_INITIAL_RESPONSE_BUFFER_SIZE); + if (chunk.memory == NULL) { + log_error("%s: Failed to allocate memory for curl response buffer", context); + goto cleanup; + } + chunk.size = 0; + chunk.capacity = API_DETECTION_INITIAL_RESPONSE_BUFFER_SIZE; + chunk.memory[0] = '\0'; + + curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_memory_callback); + curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *)&chunk); + curl_easy_setopt(curl, CURLOPT_TIMEOUT, API_DETECTION_TIMEOUT_SECONDS); + setup_common_curl_options(curl); + + CURLcode res = curl_easy_perform(curl); + if (res != CURLE_OK) { + log_error("%s: curl_easy_perform() failed: %s", context, curl_easy_strerror(res)); + log_tls_error_details(context, curl, res, request_url); + + if (res == CURLE_COULDNT_CONNECT) { + log_error("%s: Could not connect to server at %s. Is the API server running?", context, request_url); + } else if (res == CURLE_OPERATION_TIMEDOUT) { + log_error("%s: Connection to %s timed out. Server might be slow or unreachable.", context, request_url); + } else if (res == CURLE_COULDNT_RESOLVE_HOST) { + log_error("%s: Could not resolve host %s. Check your network connection and DNS settings.", context, request_url); + } + goto cleanup; + } + + long http_code = 0; + curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &http_code); + if (http_code != 200) { + make_response_preview(&chunk, preview, sizeof(preview)); + log_error("%s: API request failed with HTTP code %ld (format: %s, response: %s)", + context, http_code, api_detection_format_name(options->format), preview); + goto cleanup; + } + + if (!chunk.memory || chunk.size == 0) { + log_error("%s: Empty response from server", context); + goto cleanup; + } + + make_response_preview(&chunk, preview, sizeof(preview)); + log_info("%s: Response preview: %s", context, preview); + + ret = api_detection_parse_response(chunk.memory, options, result); + if (ret != 0) { + log_error("%s: Response size: %zu bytes", context, chunk.size); + } + +cleanup: + free(body); + free(chunk.memory); + curl_mime_free(mime); + curl_slist_free_all(headers); + curl_easy_cleanup(curl); + return ret; +} + +/** + * Apply zone and object filters, persist, and publish detections for a stream. + * Returns 0 on success, -1 when zone filtering fails. + */ +static int finalize_detections(const char *context, const char *stream_name, + detection_result_t *result, time_t frame_timestamp, + uint64_t recording_id) { + if (!stream_name || stream_name[0] == '\0') { + log_warn("%s: No stream name provided, skipping database storage", context); + return 0; + } + + log_info("%s: Filtering %d detections by zones for stream %s", context, result->count, stream_name); + if (filter_detections_by_zones(stream_name, result) != 0) { + log_error("%s: Failed to filter detections by zones for stream %s, aborting detection pipeline for this frame", + context, stream_name); + return -1; + } + + filter_detections_by_stream_objects(stream_name, result); + + time_t timestamp = (frame_timestamp != 0) ? frame_timestamp : time(NULL); + store_detections_in_db(stream_name, result, timestamp, recording_id); + + if (result->count > 0) { + char event_error[256] = {0}; + if (event_producer_publish_detection_for_stream( + stream_name, result, timestamp, + event_error, sizeof(event_error)) != 0) { + log_debug("%s: Event enqueue failed for %s: %s", context, stream_name, event_error); + } + } + + return 0; +} + +/* ------------------------------------------------------------------------- */ +/* Lifecycle */ +/* ------------------------------------------------------------------------- */ + /** * Initialize the API detection system */ @@ -363,59 +885,52 @@ void shutdown_api_detection_system(void) { log_info("API detection system shutdown complete"); } -/** - * Detect objects using the API with go2rtc snapshot - */ -int detect_objects_api(const char *api_url, const unsigned char *frame_data, - int width, int height, int channels, detection_result_t *result, - const char *stream_name, float threshold, uint64_t recording_id, - time_t frame_timestamp) { +/* ------------------------------------------------------------------------- */ +/* Entry points */ +/* ------------------------------------------------------------------------- */ + +int detect_objects_api_with_options(const char *api_url, const unsigned char *frame_data, + int width, int height, int channels, + detection_result_t *result, const char *stream_name, + float threshold, uint64_t recording_id, + time_t frame_timestamp, + const api_detection_options_t *options) { + static const char *context = "API Detection"; + // Check if we're in shutdown mode or if the stream has been stopped. if (is_shutdown_initiated()) { - log_info("API Detection: System shutdown in progress, skipping detection"); + log_info("%s: System shutdown in progress, skipping detection", context); return -1; } // Initialize result to empty at the beginning to prevent segmentation faults. - if (result) { - memset(result, 0, sizeof(detection_result_t)); - } else { - log_error("API Detection: NULL result pointer provided"); + if (!result) { + log_error("%s: NULL result pointer provided", context); return -1; } + memset(result, 0, sizeof(detection_result_t)); - // Check if api_url is the special "api-detection" string. - // If so, get the actual URL from the global config. - const char *actual_api_url = api_url; - if (api_url && strcmp(api_url, "api-detection") == 0) { - actual_api_url = g_config.api_detection_url; - log_info("API Detection: Using API URL from config: %s", actual_api_url ? actual_api_url : "NULL"); - } - - log_info("API Detection: Starting detection with API URL: %s", actual_api_url); - log_info("API Detection: Stream name: %s", stream_name ? stream_name : "NULL"); + const char *actual_api_url = resolve_api_url(api_url, context); + log_info("%s: Starting detection with API URL: %s", context, actual_api_url ? actual_api_url : "NULL"); + log_info("%s: Stream name: %s", context, stream_name ? stream_name : "NULL"); if (!is_api_detection_system_initialized()) { log_error("API detection system not initialized"); return -1; } - if (!validate_api_detection_base_url(actual_api_url, "API Detection")) { + if (!validate_api_detection_base_url(actual_api_url, context)) { return -1; } + api_detection_options_t storage; + options = resolve_options(options, &storage); + // Use go2rtc to get a JPEG snapshot directly only when we do not already // have a decoded frame. This avoids re-entering the go2rtc snapshot path // during fallback flows that already decoded a local frame. unsigned char *jpeg_data = NULL; size_t jpeg_size = 0; - CURL *local_curl = NULL; - curl_mime *mime = NULL; - curl_mimepart *part = NULL; - memory_struct_t chunk = {0}; - struct curl_slist *headers = NULL; - cJSON *root = NULL; - int ret = -1; bool go2rtc_initialized = false; bool snapshot_ok = false; @@ -427,14 +942,14 @@ int detect_objects_api(const char *api_url, const unsigned char *frame_data, } if (snapshot_ok) { - log_info("API Detection: Successfully fetched snapshot from go2rtc: %zu bytes", jpeg_size); + log_info("%s: Successfully fetched snapshot from go2rtc: %zu bytes", context, jpeg_size); } else { if (!stream_name || stream_name[0] == '\0') { - log_debug("API Detection: No stream name provided for go2rtc snapshot, using cached JPEG encoding"); + log_debug("%s: No stream name provided for go2rtc snapshot, using cached JPEG encoding", context); } else if (!go2rtc_initialized) { - log_debug("API Detection: go2rtc not initialized, using cached JPEG encoding"); + log_debug("%s: go2rtc not initialized, using cached JPEG encoding", context); } else { - log_warn("API Detection: Failed to get snapshot from go2rtc, falling back to cached JPEG encoding"); + log_warn("%s: Failed to get snapshot from go2rtc, falling back to cached JPEG encoding", context); } // FALLBACK: Use cached JPEG encoder to encode raw frame to JPEG in memory. @@ -449,304 +964,50 @@ int detect_objects_api(const char *api_url, const unsigned char *frame_data, // cache-clear in the encoder module); there is no per-call teardown here. jpeg_encoder_cache_t *encoder = jpeg_encoder_get_cached(width, height, channels, API_DETECTION_JPEG_QUALITY_DEFAULT); if (!encoder) { - log_error("API Detection: Failed to get cached JPEG encoder"); - goto cleanup; + log_error("%s: Failed to get cached JPEG encoder", context); + return -1; } // Encode directly to memory - no temp file needed - int encode_result = jpeg_encoder_cache_encode_to_memory(encoder, frame_data, &jpeg_data, &jpeg_size); - if (encode_result != 0) { - log_error("API Detection: Failed to encode frame to JPEG using cached encoder"); - goto cleanup; + if (jpeg_encoder_cache_encode_to_memory(encoder, frame_data, &jpeg_data, &jpeg_size) != 0) { + log_error("%s: Failed to encode frame to JPEG using cached encoder", context); + return -1; } - log_info("API Detection: Encoded frame to JPEG using cached encoder: %zu bytes", jpeg_size); + log_info("%s: Encoded frame to JPEG using cached encoder: %zu bytes", context, jpeg_size); } // Validate JPEG data. if (!jpeg_data || jpeg_size == 0) { - log_error("API Detection: No JPEG data available"); - goto cleanup; + log_error("%s: No JPEG data available", context); + free(jpeg_data); + return -1; } - // Use a per-call curl handle so detection requests can run concurrently. - local_curl = curl_easy_init(); - if (local_curl == NULL) { - log_error("API Detection: Failed to initialize CURL handle"); - goto cleanup; + int ret = perform_detection_request(context, actual_api_url, jpeg_data, jpeg_size, + threshold, stream_name, options, result); + free(jpeg_data); + + if (ret == 0) { + ret = finalize_detections(context, stream_name, result, frame_timestamp, recording_id); } - // Set up curl for multipart/form-data using the modern mime API. - // Note: curl_mime_* replaced deprecated curl_formadd in libcurl 7.56.0. - mime = curl_mime_init(local_curl); - if (!mime) { - log_error("API Detection: Failed to create mime structure"); - goto cleanup; - } - - part = curl_mime_addpart(mime); - if (!part) { - log_error("API Detection: Failed to add mime part"); - goto cleanup; - } - - CURLcode mime_result; - mime_result = curl_mime_name(part, "file"); - if (mime_result != CURLE_OK) { - log_error("API Detection: Failed to set mime name: %s", curl_easy_strerror(mime_result)); - goto cleanup; - } - - // Use curl_mime_data to pass data directly from memory (CURL_ZERO_TERMINATED not used; we pass size). - // curl_mime_data copies the data, so jpeg_data can be freed after this call. - mime_result = curl_mime_data(part, (const char *)jpeg_data, jpeg_size); - if (mime_result != CURLE_OK) { - log_error("API Detection: Failed to set mime data: %s", curl_easy_strerror(mime_result)); - goto cleanup; - } - - mime_result = curl_mime_filename(part, "snapshot.jpg"); - if (mime_result != CURLE_OK) { - log_error("API Detection: Failed to set mime filename: %s", curl_easy_strerror(mime_result)); - goto cleanup; - } - - mime_result = curl_mime_type(part, "image/jpeg"); - if (mime_result != CURLE_OK) { - log_error("API Detection: Failed to set mime type: %s", curl_easy_strerror(mime_result)); - goto cleanup; - } - - // Free the JPEG data now that curl has copied it. - free(jpeg_data); - jpeg_data = NULL; - - log_info("API Detection: Successfully added JPEG data to form (%zu bytes)", jpeg_size); - - const char *backend = g_config.api_detection_backend; - const char *safe_backend = sanitize_backend(backend); - float actual_threshold = normalize_api_detection_threshold(threshold); - - char url_with_params[1024]; - if (build_api_detection_url(url_with_params, - sizeof(url_with_params), - actual_api_url, - backend, - threshold, - false) != 0) { - log_error("API Detection: Failed to construct URL with parameters."); - goto cleanup; - } - log_info("API Detection: Using URL with parameters: %s (backend: %s, threshold: %.2f)", - url_with_params, safe_backend, actual_threshold); - - curl_easy_setopt(local_curl, CURLOPT_URL, url_with_params); - curl_easy_setopt(local_curl, CURLOPT_MIMEPOST, mime); - - headers = curl_slist_append(headers, "accept: application/json"); - curl_easy_setopt(local_curl, CURLOPT_HTTPHEADER, headers); - - chunk.memory = malloc(API_DETECTION_INITIAL_RESPONSE_BUFFER_SIZE); - if (chunk.memory == NULL) { - log_error("API Detection: Failed to allocate memory for curl response buffer"); - goto cleanup; - } - chunk.size = 0; - chunk.capacity = API_DETECTION_INITIAL_RESPONSE_BUFFER_SIZE; - chunk.memory[0] = '\0'; - - curl_easy_setopt(local_curl, CURLOPT_WRITEFUNCTION, write_memory_callback); - curl_easy_setopt(local_curl, CURLOPT_WRITEDATA, (void *)&chunk); - - curl_easy_setopt(local_curl, CURLOPT_TIMEOUT, API_DETECTION_TIMEOUT_SECONDS); - setup_common_curl_options(local_curl); - - log_info("API Detection: Sending request to %s", url_with_params); - - CURLcode res = curl_easy_perform(local_curl); - - if (res != CURLE_OK) { - log_error("API Detection: curl_easy_perform() failed: %s", curl_easy_strerror(res)); - log_tls_error_details("API Detection", local_curl, res, url_with_params); - - if (res == CURLE_COULDNT_CONNECT) { - log_error("API Detection: Could not connect to server at %s. Is the API server running?", url_with_params); - } else if (res == CURLE_OPERATION_TIMEDOUT) { - log_error("API Detection: Connection to %s timed out. Server might be slow or unreachable.", url_with_params); - } else if (res == CURLE_COULDNT_RESOLVE_HOST) { - log_error("API Detection: Could not resolve host %s. Check your network connection and DNS settings.", url_with_params); - } - - goto cleanup; - } - - long http_code = 0; - curl_easy_getinfo(local_curl, CURLINFO_RESPONSE_CODE, &http_code); - - if (http_code != 200) { - log_error("API request failed with HTTP code %ld", http_code); - goto cleanup; - } - - if (!chunk.memory || chunk.size == 0) { - log_error("API Detection: Empty response from server"); - goto cleanup; - } - - char preview[API_DETECTION_RESPONSE_PREVIEW_LEN]; - int preview_len = (int)(chunk.size < (API_DETECTION_RESPONSE_PREVIEW_LEN - 1) - ? chunk.size - : (API_DETECTION_RESPONSE_PREVIEW_LEN - 1)); - memcpy(preview, chunk.memory, preview_len); - preview[preview_len] = '\0'; - // Replace non-printable characters with dots - for (int i = 0; i < preview_len; i++) { - if (preview[i] < ASCII_PRINTABLE_MIN || preview[i] > ASCII_PRINTABLE_MAX) { - preview[i] = '.'; - } - } - log_info("API Detection: Response preview: %s", preview); - - root = cJSON_Parse(chunk.memory); - - if (!root) { - const char *error_ptr = cJSON_GetErrorPtr(); - log_error("Failed to parse JSON response: %s", error_ptr ? error_ptr : "Unknown error"); - log_error("API Detection: Response size: %zu bytes", chunk.size); - log_error("API Detection: Response preview: %s", preview); - goto cleanup; - } - - cJSON *detections = cJSON_GetObjectItem(root, "detections"); - if (!detections || !cJSON_IsArray(detections)) { - log_error("Invalid JSON response: missing or invalid 'detections' array"); - char *json_str = cJSON_Print(root); - if (json_str) { - log_error("API Detection: Full JSON response: %s", json_str); - free(json_str); - } - goto cleanup; - } - - int array_size = cJSON_GetArraySize(detections); - for (int i = 0; i < array_size; i++) { - if (result->count >= MAX_DETECTIONS) { - log_warn("Maximum number of detections reached (%d)", MAX_DETECTIONS); - break; - } - - cJSON *detection = cJSON_GetArrayItem(detections, i); - if (!detection) continue; - - // Extract the detection data - cJSON *label = cJSON_GetObjectItem(detection, "label"); - cJSON *confidence = cJSON_GetObjectItem(detection, "confidence"); - - // The bounding box coordinates might be in a nested object - cJSON *bounding_box = cJSON_GetObjectItem(detection, "bounding_box"); - cJSON *x_min = NULL; - cJSON *y_min = NULL; - cJSON *x_max = NULL; - cJSON *y_max = NULL; - - if (bounding_box) { - x_min = cJSON_GetObjectItem(bounding_box, "x_min"); - y_min = cJSON_GetObjectItem(bounding_box, "y_min"); - x_max = cJSON_GetObjectItem(bounding_box, "x_max"); - y_max = cJSON_GetObjectItem(bounding_box, "y_max"); - log_info("API Detection: Found bounding_box object in JSON response"); - } else { - x_min = cJSON_GetObjectItem(detection, "x_min"); - y_min = cJSON_GetObjectItem(detection, "y_min"); - x_max = cJSON_GetObjectItem(detection, "x_max"); - y_max = cJSON_GetObjectItem(detection, "y_max"); - log_info("API Detection: Using direct coordinates from JSON response"); - } - - if (!label || !cJSON_IsString(label) || - !confidence || !cJSON_IsNumber(confidence) || - !x_min || !cJSON_IsNumber(x_min) || - !y_min || !cJSON_IsNumber(y_min) || - !x_max || !cJSON_IsNumber(x_max) || - !y_max || !cJSON_IsNumber(y_max)) { - log_warn("Invalid detection data in JSON response"); - char *json_str = cJSON_Print(detection); - if (json_str) { - log_warn("Detection JSON: %s", json_str); - free(json_str); - } - continue; - } - - // Add the detection to the result - safe_strcpy(result->detections[result->count].label, label->valuestring, MAX_LABEL_LENGTH, 0); - result->detections[result->count].confidence = (float)confidence->valuedouble; - result->detections[result->count].x = (float)x_min->valuedouble; - result->detections[result->count].y = (float)y_min->valuedouble; - result->detections[result->count].width = (float)(x_max->valuedouble - x_min->valuedouble); - result->detections[result->count].height = (float)(y_max->valuedouble - y_min->valuedouble); - - // Parse optional track_id field - cJSON *track_id = cJSON_GetObjectItem(detection, "track_id"); - if (track_id && cJSON_IsNumber(track_id)) { - result->detections[result->count].track_id = (int)track_id->valuedouble; - } else { - result->detections[result->count].track_id = -1; // No tracking - } - - // Parse optional zone_id field - cJSON *zone_id = cJSON_GetObjectItem(detection, "zone_id"); - if (zone_id && cJSON_IsString(zone_id)) { - safe_strcpy(result->detections[result->count].zone_id, zone_id->valuestring, MAX_ZONE_ID_LENGTH, 0); - } else { - result->detections[result->count].zone_id[0] = '\0'; // Empty zone - } - - result->count++; - } - - // Filter detections by zones before storing - if (stream_name && stream_name[0] != '\0') { - log_info("API Detection: Filtering %d detections by zones for stream %s", result->count, stream_name); - int filter_ret = filter_detections_by_zones(stream_name, result); - if (filter_ret != 0) { - log_error("Failed to filter detections by zones, aborting detection pipeline for this frame"); - goto cleanup; - } - - filter_detections_by_stream_objects(stream_name, result); - - time_t timestamp = (frame_timestamp != 0) ? frame_timestamp : time(NULL); - store_detections_in_db(stream_name, result, timestamp, recording_id); - - if (result->count > 0) { - char event_error[256] = {0}; - if (event_producer_publish_detection_for_stream( - stream_name, result, timestamp, - event_error, sizeof(event_error)) != 0) { - log_debug("API Detection: Event enqueue failed for %s: %s", - stream_name, event_error); - } - } - } else { - log_warn("No stream name provided, skipping database storage"); - } - - ret = 0; - -cleanup: - cJSON_Delete(root); - free(chunk.memory); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); - free(jpeg_data); - if (ret != 0) { result->count = 0; + return ret; } - return ret; + log_info("%s: Successfully detected %d objects", context, result->count); + return 0; +} + +int detect_objects_api(const char *api_url, const unsigned char *frame_data, + int width, int height, int channels, detection_result_t *result, + const char *stream_name, float threshold, uint64_t recording_id, + time_t frame_timestamp) { + return detect_objects_api_with_options(api_url, frame_data, width, height, channels, + result, stream_name, threshold, recording_id, + frame_timestamp, NULL); } /** @@ -758,357 +1019,88 @@ int detect_objects_api(const char *api_url, const unsigned char *frame_data, * * Returns: 0 on success, -1 on general failure, -2 if go2rtc snapshot failed */ -int detect_objects_api_snapshot(const char *api_url, const char *stream_name, - detection_result_t *result, float threshold, - uint64_t recording_id, time_t frame_timestamp) { +int detect_objects_api_snapshot_with_options(const char *api_url, const char *stream_name, + detection_result_t *result, float threshold, + uint64_t recording_id, time_t frame_timestamp, + const api_detection_options_t *options) { + static const char *context = "API Detection (snapshot)"; + // Check if we're in shutdown mode if (is_shutdown_initiated()) { - log_info("API Detection (snapshot): System shutdown in progress, skipping detection"); + log_info("%s: System shutdown in progress, skipping detection", context); return -1; } // Stream name is required for go2rtc snapshot if (!stream_name || stream_name[0] == '\0') { - log_error("API Detection (snapshot): Stream name is required"); + log_error("%s: Stream name is required", context); return -1; } // Initialize result - if (result) { - memset(result, 0, sizeof(detection_result_t)); - } else { - log_error("API Detection (snapshot): NULL result pointer provided"); + if (!result) { + log_error("%s: NULL result pointer provided", context); return -1; } + memset(result, 0, sizeof(detection_result_t)); - // Handle "api-detection" special string - const char *actual_api_url = api_url; - if (api_url && strcmp(api_url, "api-detection") == 0) { - actual_api_url = g_config.api_detection_url; - log_info("API Detection (snapshot): Using API URL from config: %s", actual_api_url ? actual_api_url : "NULL"); - } + const char *actual_api_url = resolve_api_url(api_url, context); if (!is_api_detection_system_initialized()) { log_error("API detection system not initialized"); return -1; } - if (!validate_api_detection_base_url(actual_api_url, "API Detection (snapshot)")) { + if (!validate_api_detection_base_url(actual_api_url, context)) { return -1; } + api_detection_options_t storage; + options = resolve_options(options, &storage); + // Try to get snapshot from go2rtc (only if go2rtc is initialized) unsigned char *jpeg_data = NULL; size_t jpeg_size = 0; if (!go2rtc_integration_is_initialized()) { - log_debug("API Detection (snapshot): go2rtc not initialized, skipping snapshot for stream %s", stream_name); - return -2; // Special return code: go2rtc not available, caller should fall back + log_debug("%s: go2rtc not initialized, skipping snapshot for stream %s", context, stream_name); + return API_DETECTION_SNAPSHOT_UNAVAILABLE; // caller should fall back } if (!go2rtc_get_snapshot(stream_name, &jpeg_data, &jpeg_size)) { - log_warn("API Detection (snapshot): Failed to get snapshot from go2rtc for stream %s", stream_name); - return -2; // Special return code: go2rtc failed, caller should fall back + log_warn("%s: Failed to get snapshot from go2rtc for stream %s", context, stream_name); + return API_DETECTION_SNAPSHOT_UNAVAILABLE; // caller should fall back } - log_info("API Detection (snapshot): Successfully fetched snapshot from go2rtc: %zu bytes", jpeg_size); + log_info("%s: Successfully fetched snapshot from go2rtc: %zu bytes", context, jpeg_size); // Validate JPEG data if (!jpeg_data || jpeg_size == 0) { - log_error("API Detection (snapshot): No JPEG data available"); - if (jpeg_data) free(jpeg_data); - return -2; - } - - // Create a per-call curl handle to allow parallel requests from multiple threads - // This avoids the global mutex bottleneck that was serializing all detection calls - CURL *local_curl = curl_easy_init(); - if (!local_curl) { - log_error("API Detection (snapshot): Failed to create curl handle"); - free(jpeg_data); - return -1; - } - - // Set up curl for multipart/form-data - curl_mime *mime = NULL; - curl_mimepart *part = NULL; - memory_struct_t chunk = {0}; - chunk.memory = NULL; - struct curl_slist *headers = NULL; - - // Create the mime structure - mime = curl_mime_init(local_curl); - if (!mime) { - log_error("API Detection (snapshot): Failed to create mime structure"); - free(jpeg_data); - curl_easy_cleanup(local_curl); - return -1; - } - - // Add the file part - part = curl_mime_addpart(mime); - if (!part) { - log_error("API Detection (snapshot): Failed to add mime part"); - curl_mime_free(mime); - free(jpeg_data); - curl_easy_cleanup(local_curl); - return -1; - } - - // Set up the mime part - CURLcode mime_result; - mime_result = curl_mime_name(part, "file"); - if (mime_result != CURLE_OK) { - log_error("API Detection (snapshot): curl_mime_name failed: %s", - curl_easy_strerror(mime_result)); - curl_mime_free(mime); - free(jpeg_data); - curl_easy_cleanup(local_curl); - return -1; - } - - mime_result = curl_mime_data(part, (const char *)jpeg_data, jpeg_size); - if (mime_result != CURLE_OK) { - log_error("API Detection (snapshot): curl_mime_data failed: %s", - curl_easy_strerror(mime_result)); - curl_mime_free(mime); - free(jpeg_data); - curl_easy_cleanup(local_curl); - return -1; - } - - mime_result = curl_mime_filename(part, "snapshot.jpg"); - if (mime_result != CURLE_OK) { - log_error("API Detection (snapshot): curl_mime_filename failed: %s", - curl_easy_strerror(mime_result)); - curl_mime_free(mime); + log_error("%s: No JPEG data available", context); free(jpeg_data); - curl_easy_cleanup(local_curl); - return -1; + return API_DETECTION_SNAPSHOT_UNAVAILABLE; } - mime_result = curl_mime_type(part, "image/jpeg"); - if (mime_result != CURLE_OK) { - log_error("API Detection (snapshot): curl_mime_type failed: %s", - curl_easy_strerror(mime_result)); - curl_mime_free(mime); - free(jpeg_data); - curl_easy_cleanup(local_curl); - return -1; - } - - // Free JPEG data now that curl has copied it + int ret = perform_detection_request(context, actual_api_url, jpeg_data, jpeg_size, + threshold, stream_name, options, result); free(jpeg_data); - jpeg_data = NULL; - - const char *backend = g_config.api_detection_backend; - const char *sanitized_backend = sanitize_backend(backend); - float actual_threshold = normalize_api_detection_threshold(threshold); - - char url_with_params[1024]; - if (build_api_detection_url(url_with_params, - sizeof(url_with_params), - actual_api_url, - backend, - threshold, - false) != 0) { - log_error("API Detection (snapshot): URL too long when constructing request"); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); - return -1; - } - - log_info("API Detection (snapshot): Sending request to %s (backend: %s, threshold: %.2f)", - url_with_params, - sanitized_backend, - actual_threshold); - - // Set up the request - curl_easy_setopt(local_curl, CURLOPT_URL, url_with_params); - curl_easy_setopt(local_curl, CURLOPT_MIMEPOST, mime); - headers = curl_slist_append(headers, "accept: application/json"); - curl_easy_setopt(local_curl, CURLOPT_HTTPHEADER, headers); - - chunk.memory = malloc(API_DETECTION_INITIAL_RESPONSE_BUFFER_SIZE); - if (chunk.memory == NULL) { - log_error("API Detection (snapshot): Failed to allocate memory for curl response buffer"); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); - return -1; - } - chunk.size = 0; - chunk.capacity = API_DETECTION_INITIAL_RESPONSE_BUFFER_SIZE; - chunk.memory[0] = '\0'; - - curl_easy_setopt(local_curl, CURLOPT_WRITEFUNCTION, write_memory_callback); - curl_easy_setopt(local_curl, CURLOPT_WRITEDATA, (void *)&chunk); - curl_easy_setopt(local_curl, CURLOPT_TIMEOUT, API_DETECTION_TIMEOUT_SECONDS); - setup_common_curl_options(local_curl); - - // Perform the request - CURLcode res = curl_easy_perform(local_curl); - - if (res != CURLE_OK) { - log_error("API Detection (snapshot): curl_easy_perform() failed: %s", curl_easy_strerror(res)); - log_tls_error_details("API Detection (snapshot)", local_curl, res, url_with_params); - free(chunk.memory); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); - return -1; + if (ret == 0) { + ret = finalize_detections(context, stream_name, result, frame_timestamp, recording_id); } - // Check HTTP response code - long http_code = 0; - curl_easy_getinfo(local_curl, CURLINFO_RESPONSE_CODE, &http_code); - - if (http_code != 200) { - log_error("API Detection (snapshot): HTTP error %ld", http_code); - free(chunk.memory); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); - return -1; - } - - // Parse JSON response - if (!chunk.memory || chunk.size == 0) { - log_error("API Detection (snapshot): Empty response"); - free(chunk.memory); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); - return -1; - } - - cJSON *root = cJSON_Parse(chunk.memory); - if (!root) { - log_error("API Detection (snapshot): Failed to parse JSON response"); - free(chunk.memory); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); - return -1; - } - - // Extract detections - cJSON *detections = cJSON_GetObjectItem(root, "detections"); - if (!detections || !cJSON_IsArray(detections)) { - log_error("API Detection (snapshot): Invalid JSON response"); - cJSON_Delete(root); - free(chunk.memory); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); + if (ret != 0) { + result->count = 0; return -1; } - // Process each detection - int array_size = cJSON_GetArraySize(detections); - for (int i = 0; i < array_size; i++) { - if (result->count >= MAX_DETECTIONS) { - log_warn("API Detection (snapshot): Maximum detections reached"); - break; - } - - cJSON *detection = cJSON_GetArrayItem(detections, i); - if (!detection) continue; - - cJSON *label = cJSON_GetObjectItem(detection, "label"); - cJSON *confidence = cJSON_GetObjectItem(detection, "confidence"); - - cJSON *bounding_box = cJSON_GetObjectItem(detection, "bounding_box"); - cJSON *x_min = NULL, *y_min = NULL, *x_max = NULL, *y_max = NULL; - - if (bounding_box) { - x_min = cJSON_GetObjectItem(bounding_box, "x_min"); - y_min = cJSON_GetObjectItem(bounding_box, "y_min"); - x_max = cJSON_GetObjectItem(bounding_box, "x_max"); - y_max = cJSON_GetObjectItem(bounding_box, "y_max"); - } else { - x_min = cJSON_GetObjectItem(detection, "x_min"); - y_min = cJSON_GetObjectItem(detection, "y_min"); - x_max = cJSON_GetObjectItem(detection, "x_max"); - y_max = cJSON_GetObjectItem(detection, "y_max"); - } - - if (!label || !cJSON_IsString(label) || - !confidence || !cJSON_IsNumber(confidence) || - !x_min || !cJSON_IsNumber(x_min) || - !y_min || !cJSON_IsNumber(y_min) || - !x_max || !cJSON_IsNumber(x_max) || - !y_max || !cJSON_IsNumber(y_max)) { - continue; - } - - // Add detection to result - safe_strcpy(result->detections[result->count].label, label->valuestring, MAX_LABEL_LENGTH, 0); - result->detections[result->count].confidence = (float)confidence->valuedouble; - result->detections[result->count].x = (float)x_min->valuedouble; - result->detections[result->count].y = (float)y_min->valuedouble; - result->detections[result->count].width = (float)(x_max->valuedouble - x_min->valuedouble); - result->detections[result->count].height = (float)(y_max->valuedouble - y_min->valuedouble); - - cJSON *track_id = cJSON_GetObjectItem(detection, "track_id"); - result->detections[result->count].track_id = (track_id && cJSON_IsNumber(track_id)) - ? (int)track_id->valuedouble : -1; - - cJSON *zone_id = cJSON_GetObjectItem(detection, "zone_id"); - if (zone_id && cJSON_IsString(zone_id)) { - safe_strcpy(result->detections[result->count].zone_id, zone_id->valuestring, MAX_ZONE_ID_LENGTH, 0); - } else { - result->detections[result->count].zone_id[0] = '\0'; - } - - result->count++; - } - - // Filter by zones and store in database - if (stream_name && stream_name[0] != '\0') { - log_info("API Detection (snapshot): Filtering %d detections by zones for stream %s", - result->count, stream_name); - if (filter_detections_by_zones(stream_name, result) != 0) { - log_error("API Detection (snapshot): Failed to filter detections by zones for stream %s", - stream_name); - - // Clean up on error to avoid leaking resources - cJSON_Delete(root); - free(chunk.memory); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); - - return -1; - } - - // Filter detections by per-stream object include/exclude lists - filter_detections_by_stream_objects(stream_name, result); - - time_t timestamp = (frame_timestamp != 0) ? frame_timestamp : time(NULL); - store_detections_in_db(stream_name, result, timestamp, recording_id); - - // Publish asynchronously through the normalized event bus. - if (result->count > 0) { - char event_error[256] = {0}; - if (event_producer_publish_detection_for_stream( - stream_name, result, timestamp, - event_error, sizeof(event_error)) != 0) { - log_debug("API Detection (snapshot): Event enqueue failed for %s: %s", - stream_name, event_error); - } - } - } - - // Clean up - cJSON_Delete(root); - free(chunk.memory); - curl_mime_free(mime); - curl_slist_free_all(headers); - curl_easy_cleanup(local_curl); - - log_info("API Detection (snapshot): Successfully detected %d objects", result->count); + log_info("%s: Successfully detected %d objects", context, result->count); return 0; } + +int detect_objects_api_snapshot(const char *api_url, const char *stream_name, + detection_result_t *result, float threshold, + uint64_t recording_id, time_t frame_timestamp) { + return detect_objects_api_snapshot_with_options(api_url, stream_name, result, threshold, + recording_id, frame_timestamp, NULL); +} diff --git a/src/video/unified_detection_thread.c b/src/video/unified_detection_thread.c index 3ad24148..635d3834 100644 --- a/src/video/unified_detection_thread.c +++ b/src/video/unified_detection_thread.c @@ -167,6 +167,31 @@ static const char *get_actual_api_url(const char *stream_name, const char *model } return actual_api_url; } + +/** + * Resolve the request options for the stream's API detector: the global + * [api_detection] settings first, then per-engine overrides from the engine's + * config JSON ({"format","backend","detector_name"}) when the stream's only + * engine is an api engine. That is the single-engine dispatch path, which is + * the only place an API engine actually runs today. + */ +static void resolve_api_detection_options(const unified_detection_ctx_t *ctx, + api_detection_options_t *options) +{ + api_detection_options_from_config(options); + if (!ctx || ctx->engine_count != 1) { + return; + } + const stream_detection_engine_t *engine = &ctx->engines[0]; + if (strcmp(engine->engine_type, "api") != 0 || engine->config_json[0] == '\0') { + return; + } + if (api_detection_options_apply_json(options, engine->config_json) != 0) { + log_warn("[%s] Detection engine %s has an invalid api detection config; using global settings", + ctx->stream_name, engine->engine_key); + api_detection_options_from_config(options); + } +} /** * Helper to update stored video parameters, distinguishing between * container-derived FPS values and provisional fallbacks that should @@ -2842,9 +2867,11 @@ static bool run_detection_on_frame(unified_detection_ctx_t *ctx, AVPacket *pkt, detection_result_t result; memset(&result, 0, sizeof(result)); - int detect_ret = detect_objects_api_snapshot(ctx->model_path, ctx->stream_name, - &result, ctx->detection_threshold, - rec_id, frame_timestamp); + api_detection_options_t api_options; + resolve_api_detection_options(ctx, &api_options); + int detect_ret = detect_objects_api_snapshot_with_options( + ctx->model_path, ctx->stream_name, &result, ctx->detection_threshold, + rec_id, frame_timestamp, &api_options); if (detect_ret != DETECT_SNAPSHOT_UNAVAILABLE) { if (detect_ret != 0) { @@ -3056,10 +3083,13 @@ static bool detect_on_decoded_frame(unified_detection_ctx_t *ctx, if (!rgb_buf) return false; const char *api_url = get_actual_api_url(ctx->stream_name, ctx->model_path); + api_detection_options_t api_options; + resolve_api_detection_options(ctx, &api_options); int ret = api_url - ? detect_objects_api(api_url, rgb_buf, width, height, 3, - result, ctx->stream_name, ctx->detection_threshold, - rec_id, now) + ? detect_objects_api_with_options(api_url, rgb_buf, width, height, 3, + result, ctx->stream_name, + ctx->detection_threshold, rec_id, now, + &api_options) : -1; free(rgb_buf); diff --git a/src/web/api_handlers_settings.c b/src/web/api_handlers_settings.c index f88a813d..e80e3758 100644 --- a/src/web/api_handlers_settings.c +++ b/src/web/api_handlers_settings.c @@ -790,6 +790,8 @@ void handle_get_settings(const http_request_t *req, http_response_t *res) { // API detection settings cJSON_AddStringToObject(settings, "api_detection_url", g_config.api_detection_url); cJSON_AddStringToObject(settings, "api_detection_backend", g_config.api_detection_backend); + cJSON_AddStringToObject(settings, "api_detection_format", g_config.api_detection_format); + cJSON_AddStringToObject(settings, "api_detection_detector_name", g_config.api_detection_detector_name); // Detection defaults cJSON_AddNumberToObject(settings, "default_detection_threshold", g_config.default_detection_threshold); @@ -955,6 +957,20 @@ void handle_post_settings(const http_request_t *req, http_response_t *res) { return; } + // Preflight: reject an unknown api_detection_format before any setting is + // applied, so a bad request cannot leave g_config partially updated. + { + const cJSON *format = cJSON_GetObjectItem(settings, "api_detection_format"); + api_detection_format_t parsed_format; + if (format && cJSON_IsString(format) && + !api_detection_format_parse(format->valuestring, &parsed_format)) { + log_warn("Rejected invalid api_detection_format setting"); + cJSON_Delete(settings); + http_response_set_json_error(res, 400, "Invalid api_detection_format: expected light-object-detect or doods2"); + return; + } + } + // Update settings bool settings_changed = false; bool restart_required = false; @@ -1554,6 +1570,24 @@ void handle_post_settings(const http_request_t *req, http_response_t *res) { log_info("Updated api_detection_backend: %s", g_config.api_detection_backend); } + // API detection request format (light-object-detect or doods2); validated above. + cJSON *api_detection_format = cJSON_GetObjectItem(settings, "api_detection_format"); + if (api_detection_format && cJSON_IsString(api_detection_format) && + config_set_api_detection_format(&g_config, api_detection_format->valuestring)) { + settings_changed = true; + log_info("Updated api_detection_format: %s", g_config.api_detection_format); + } + + // DOODS2 detector name + cJSON *api_detection_detector_name = cJSON_GetObjectItem(settings, "api_detection_detector_name"); + if (api_detection_detector_name && cJSON_IsString(api_detection_detector_name)) { + const char *detector_name = api_detection_detector_name->valuestring; + safe_strcpy(g_config.api_detection_detector_name, detector_name[0] ? detector_name : "default", + sizeof(g_config.api_detection_detector_name), 0); + settings_changed = true; + log_info("Updated api_detection_detector_name: %s", g_config.api_detection_detector_name); + } + // In-process LiteRT (TFLite) detection engine settings cJSON *de_enabled = cJSON_GetObjectItem(settings, "detection_engine_enabled"); if (de_enabled && cJSON_IsBool(de_enabled)) { diff --git a/tests/unit/test_api_detection.c b/tests/unit/test_api_detection.c index 011ff14e..fc5617c1 100644 --- a/tests/unit/test_api_detection.c +++ b/tests/unit/test_api_detection.c @@ -1,18 +1,28 @@ /** * @file test_api_detection.c - * @brief Layer 2 Unity tests for API detection URL validation. + * @brief Layer 2 Unity tests for API detection URL validation, request + * construction, and response parsing for both wire formats. */ #define _POSIX_C_SOURCE 200809L +#include +#include #include +#include + #include "unity.h" +#include "core/config.h" #include "core/logger.h" #include "video/api_detection.h" void setUp(void) { TEST_ASSERT_EQUAL_INT(0, init_api_detection_system()); + snprintf(g_config.api_detection_format, sizeof(g_config.api_detection_format), "%s", + API_DETECTION_FORMAT_NAME_LIGHT_OBJECT_DETECT); + snprintf(g_config.api_detection_backend, sizeof(g_config.api_detection_backend), "onnx"); + snprintf(g_config.api_detection_detector_name, sizeof(g_config.api_detection_detector_name), "default"); } void tearDown(void) { @@ -29,6 +39,19 @@ static void assert_invalid_detection_url(const char *url) { TEST_ASSERT_EQUAL_INT(0, result.count); } +static api_detection_options_t make_options(api_detection_format_t format, + const char *backend, + const char *detector_name) { + api_detection_options_t options; + memset(&options, 0, sizeof(options)); + options.format = format; + if (backend) snprintf(options.backend, sizeof(options.backend), "%s", backend); + if (detector_name) snprintf(options.detector_name, sizeof(options.detector_name), "%s", detector_name); + return options; +} + +/* ---- URL validation (pre-existing behaviour) ---------------------------- */ + void test_detect_objects_api_rejects_url_with_space(void) { assert_invalid_detection_url("http://localhost:9001/api detect"); } @@ -57,6 +80,283 @@ void test_api_detection_skips_go2rtc_snapshot_without_stream_name(void) { TEST_ASSERT_FALSE(api_detection_should_use_go2rtc_snapshot(NULL, 0, 0, 0, "")); } +/* ---- Format names and options ------------------------------------------ */ + +void test_format_parse_accepts_known_names_and_aliases(void) { + api_detection_format_t format = API_DETECTION_FORMAT_DOODS2; + + TEST_ASSERT_TRUE(api_detection_format_parse("light-object-detect", &format)); + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, format); + TEST_ASSERT_TRUE(api_detection_format_parse("DOODS2", &format)); + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_DOODS2, format); + TEST_ASSERT_TRUE(api_detection_format_parse(" doods ", &format)); + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_DOODS2, format); + TEST_ASSERT_TRUE(api_detection_format_parse("lod", &format)); + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, format); + + format = API_DETECTION_FORMAT_DOODS2; + TEST_ASSERT_FALSE(api_detection_format_parse("deepstack", &format)); + TEST_ASSERT_FALSE(api_detection_format_parse("", &format)); + TEST_ASSERT_FALSE(api_detection_format_parse(NULL, &format)); + TEST_ASSERT_FALSE(api_detection_format_parse("doods2", NULL)); + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_DOODS2, format); + + TEST_ASSERT_EQUAL_STRING("light-object-detect", + api_detection_format_name(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT)); + TEST_ASSERT_EQUAL_STRING("doods2", api_detection_format_name(API_DETECTION_FORMAT_DOODS2)); +} + +void test_options_from_config_reads_global_settings(void) { + snprintf(g_config.api_detection_format, sizeof(g_config.api_detection_format), "doods2"); + snprintf(g_config.api_detection_backend, sizeof(g_config.api_detection_backend), "tflite"); + snprintf(g_config.api_detection_detector_name, sizeof(g_config.api_detection_detector_name), "tensorflow"); + + api_detection_options_t options; + api_detection_options_from_config(&options); + + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_DOODS2, options.format); + TEST_ASSERT_EQUAL_STRING("tflite", options.backend); + TEST_ASSERT_EQUAL_STRING("tensorflow", options.detector_name); +} + +void test_options_from_config_falls_back_on_unknown_format(void) { + snprintf(g_config.api_detection_format, sizeof(g_config.api_detection_format), "bogus"); + + api_detection_options_t options; + api_detection_options_from_config(&options); + + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, options.format); +} + +void test_options_apply_json_overrides_only_known_keys(void) { + api_detection_options_t options = make_options(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, "onnx", "default"); + + TEST_ASSERT_EQUAL_INT(0, api_detection_options_apply_json(&options, + "{\"format\":\"doods2\",\"detector_name\":\"tensorflow\",\"unrelated\":1}")); + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_DOODS2, options.format); + TEST_ASSERT_EQUAL_STRING("onnx", options.backend); + TEST_ASSERT_EQUAL_STRING("tensorflow", options.detector_name); + + TEST_ASSERT_EQUAL_INT(0, api_detection_options_apply_json(&options, "{\"backend\":\"opencv\"}")); + TEST_ASSERT_EQUAL_STRING("opencv", options.backend); + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_DOODS2, options.format); + + /* Empty, absent, and {} leave everything untouched. */ + TEST_ASSERT_EQUAL_INT(0, api_detection_options_apply_json(&options, "{}")); + TEST_ASSERT_EQUAL_INT(0, api_detection_options_apply_json(&options, "")); + TEST_ASSERT_EQUAL_INT(0, api_detection_options_apply_json(&options, NULL)); + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_DOODS2, options.format); + TEST_ASSERT_EQUAL_STRING("tensorflow", options.detector_name); +} + +void test_options_apply_json_rejects_bad_input(void) { + api_detection_options_t options = make_options(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, "onnx", "default"); + + TEST_ASSERT_EQUAL_INT(-1, api_detection_options_apply_json(&options, "{\"format\":\"deepstack\"}")); + TEST_ASSERT_EQUAL_INT(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, options.format); + TEST_ASSERT_EQUAL_INT(-1, api_detection_options_apply_json(&options, "{\"format\":7}")); + TEST_ASSERT_EQUAL_INT(-1, api_detection_options_apply_json(&options, "not json")); + TEST_ASSERT_EQUAL_INT(-1, api_detection_options_apply_json(&options, "[1,2]")); + TEST_ASSERT_EQUAL_INT(-1, api_detection_options_apply_json(NULL, "{}")); +} + +/* ---- Request construction ---------------------------------------------- */ + +void test_build_request_url_appends_light_object_detect_params(void) { + api_detection_options_t options = make_options(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, "tflite", NULL); + char url[512]; + + TEST_ASSERT_EQUAL_INT(0, api_detection_build_request_url(url, sizeof(url), + "http://host:8000/api/v1/detect", 0.35f, &options)); + TEST_ASSERT_EQUAL_STRING( + "http://host:8000/api/v1/detect?backend=tflite&confidence_threshold=0.35&return_image=false", url); + + /* An existing query string is extended, not replaced. */ + TEST_ASSERT_EQUAL_INT(0, api_detection_build_request_url(url, sizeof(url), + "http://host:8000/api/v1/detect?tiles=4", 0.35f, &options)); + TEST_ASSERT_EQUAL_STRING( + "http://host:8000/api/v1/detect?tiles=4&backend=tflite&confidence_threshold=0.35&return_image=false", url); + + /* Non-positive thresholds fall back to 0.5 and unsafe backends to onnx. */ + api_detection_options_t unsafe = make_options(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, "on nx&x=1", NULL); + TEST_ASSERT_EQUAL_INT(0, api_detection_build_request_url(url, sizeof(url), + "http://host/detect", -1.0f, &unsafe)); + TEST_ASSERT_EQUAL_STRING( + "http://host/detect?backend=onnx&confidence_threshold=0.50&return_image=false", url); +} + +void test_build_request_url_leaves_doods2_url_verbatim(void) { + api_detection_options_t options = make_options(API_DETECTION_FORMAT_DOODS2, "tflite", "default"); + char url[512]; + + TEST_ASSERT_EQUAL_INT(0, api_detection_build_request_url(url, sizeof(url), + "http://doods:8080/detect", 0.35f, &options)); + TEST_ASSERT_EQUAL_STRING("http://doods:8080/detect", url); + + TEST_ASSERT_EQUAL_INT(0, api_detection_build_request_url(url, sizeof(url), + "http://doods:8080/detect?x=1", 0.35f, &options)); + TEST_ASSERT_EQUAL_STRING("http://doods:8080/detect?x=1", url); +} + +void test_build_request_url_uses_global_config_when_options_null(void) { + snprintf(g_config.api_detection_format, sizeof(g_config.api_detection_format), "doods2"); + char url[512]; + + TEST_ASSERT_EQUAL_INT(0, api_detection_build_request_url(url, sizeof(url), + "http://doods:8080/detect", 0.35f, NULL)); + TEST_ASSERT_EQUAL_STRING("http://doods:8080/detect", url); +} + +void test_build_request_url_rejects_small_buffer_and_bad_args(void) { + api_detection_options_t options = make_options(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, "onnx", NULL); + char url[32]; + + TEST_ASSERT_EQUAL_INT(-1, api_detection_build_request_url(url, sizeof(url), + "http://host:8000/api/v1/detect", 0.5f, &options)); + TEST_ASSERT_EQUAL_INT(-1, api_detection_build_request_url(NULL, 0, "http://host", 0.5f, &options)); + TEST_ASSERT_EQUAL_INT(-1, api_detection_build_request_url(url, sizeof(url), NULL, 0.5f, &options)); + TEST_ASSERT_EQUAL_INT(-1, api_detection_build_request_url(url, sizeof(url), "http://ho st", 0.5f, &options)); +} + +void test_doods2_body_encodes_jpeg_threshold_and_detector(void) { + const unsigned char jpeg[] = {'h', 'e', 'l', 'l', 'o'}; + api_detection_options_t options = make_options(API_DETECTION_FORMAT_DOODS2, NULL, "tensorflow"); + + char *body = api_detection_build_doods2_body(jpeg, sizeof(jpeg), 0.4f, "Front \"Door\"", &options); + TEST_ASSERT_NOT_NULL(body); + + cJSON *root = cJSON_Parse(body); + TEST_ASSERT_NOT_NULL(root); + TEST_ASSERT_EQUAL_STRING("Front \"Door\"", cJSON_GetObjectItem(root, "id")->valuestring); + TEST_ASSERT_EQUAL_STRING("tensorflow", cJSON_GetObjectItem(root, "detector_name")->valuestring); + TEST_ASSERT_EQUAL_STRING("aGVsbG8=", cJSON_GetObjectItem(root, "data")->valuestring); + cJSON *detect = cJSON_GetObjectItem(root, "detect"); + TEST_ASSERT_NOT_NULL(detect); + TEST_ASSERT_FLOAT_WITHIN(0.01f, 40.0f, (float)cJSON_GetObjectItem(detect, "*")->valuedouble); + cJSON_Delete(root); + free(body); +} + +void test_doods2_body_uses_defaults_for_missing_fields(void) { + const unsigned char jpeg[] = {0xFF, 0xD8, 0xFF}; + api_detection_options_t options = make_options(API_DETECTION_FORMAT_DOODS2, NULL, NULL); + + char *body = api_detection_build_doods2_body(jpeg, sizeof(jpeg), -1.0f, NULL, &options); + TEST_ASSERT_NOT_NULL(body); + + cJSON *root = cJSON_Parse(body); + TEST_ASSERT_NOT_NULL(root); + TEST_ASSERT_EQUAL_STRING("lightnvr", cJSON_GetObjectItem(root, "id")->valuestring); + TEST_ASSERT_EQUAL_STRING("default", cJSON_GetObjectItem(root, "detector_name")->valuestring); + TEST_ASSERT_EQUAL_STRING("/9j/", cJSON_GetObjectItem(root, "data")->valuestring); + TEST_ASSERT_FLOAT_WITHIN(0.01f, 50.0f, + (float)cJSON_GetObjectItem(cJSON_GetObjectItem(root, "detect"), "*")->valuedouble); + cJSON_Delete(root); + free(body); + + TEST_ASSERT_NULL(api_detection_build_doods2_body(NULL, 0, 0.5f, NULL, &options)); + TEST_ASSERT_NULL(api_detection_build_doods2_body(jpeg, 0, 0.5f, NULL, &options)); +} + +/* ---- Response parsing -------------------------------------------------- */ + +void test_parse_response_light_object_detect_flat_and_nested_boxes(void) { + const char *json = + "{\"detections\":[" + "{\"label\":\"person\",\"confidence\":0.91,\"x_min\":0.1,\"y_min\":0.2,\"x_max\":0.4,\"y_max\":0.8}," + "{\"label\":\"car\",\"confidence\":0.5," + "\"bounding_box\":{\"x_min\":0.5,\"y_min\":0.5,\"x_max\":1.0,\"y_max\":1.0}," + "\"track_id\":7,\"zone_id\":\"drive\"}," + "{\"label\":\"broken\",\"confidence\":0.9}" + "]}"; + api_detection_options_t options = make_options(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, "onnx", NULL); + detection_result_t result; + memset(&result, 0xAB, sizeof(result)); + + TEST_ASSERT_EQUAL_INT(0, api_detection_parse_response(json, &options, &result)); + TEST_ASSERT_EQUAL_INT(2, result.count); + + TEST_ASSERT_EQUAL_STRING("person", result.detections[0].label); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.91f, result.detections[0].confidence); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.1f, result.detections[0].x); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.2f, result.detections[0].y); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.3f, result.detections[0].width); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.6f, result.detections[0].height); + TEST_ASSERT_EQUAL_INT(-1, result.detections[0].track_id); + TEST_ASSERT_EQUAL_STRING("", result.detections[0].zone_id); + + TEST_ASSERT_EQUAL_STRING("car", result.detections[1].label); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.5f, result.detections[1].x); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.5f, result.detections[1].width); + TEST_ASSERT_EQUAL_INT(7, result.detections[1].track_id); + TEST_ASSERT_EQUAL_STRING("drive", result.detections[1].zone_id); +} + +void test_parse_response_doods2_maps_boxes_and_scales_confidence(void) { + const char *json = + "{\"id\":\"Front Door\",\"detections\":[" + "{\"top\":0.2,\"left\":0.1,\"bottom\":0.8,\"right\":0.4,\"label\":\"dog\",\"confidence\":90.0}," + "{\"top\":0.0,\"left\":0.0,\"bottom\":1.0,\"right\":1.0,\"label\":\"person\",\"confidence\":120.0}" + "],\"error\":\"\"}"; + api_detection_options_t options = make_options(API_DETECTION_FORMAT_DOODS2, NULL, "default"); + detection_result_t result; + + TEST_ASSERT_EQUAL_INT(0, api_detection_parse_response(json, &options, &result)); + TEST_ASSERT_EQUAL_INT(2, result.count); + + TEST_ASSERT_EQUAL_STRING("dog", result.detections[0].label); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.9f, result.detections[0].confidence); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.1f, result.detections[0].x); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.2f, result.detections[0].y); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.3f, result.detections[0].width); + TEST_ASSERT_FLOAT_WITHIN(0.001f, 0.6f, result.detections[0].height); + TEST_ASSERT_EQUAL_INT(-1, result.detections[0].track_id); + + /* Out-of-range confidence is clamped to the 0-1 scale. */ + TEST_ASSERT_FLOAT_WITHIN(0.001f, 1.0f, result.detections[1].confidence); +} + +void test_parse_response_doods2_error_field_fails_request(void) { + api_detection_options_t options = make_options(API_DETECTION_FORMAT_DOODS2, NULL, "default"); + detection_result_t result; + + TEST_ASSERT_EQUAL_INT(-1, api_detection_parse_response( + "{\"detections\":[],\"error\":\"detector not found\"}", &options, &result)); + TEST_ASSERT_EQUAL_INT(0, result.count); +} + +void test_parse_response_handles_null_missing_and_garbage(void) { + api_detection_options_t options = make_options(API_DETECTION_FORMAT_DOODS2, NULL, "default"); + detection_result_t result; + + TEST_ASSERT_EQUAL_INT(0, api_detection_parse_response("{\"detections\":null}", &options, &result)); + TEST_ASSERT_EQUAL_INT(0, result.count); + TEST_ASSERT_EQUAL_INT(0, api_detection_parse_response("{\"detections\":[]}", &options, &result)); + TEST_ASSERT_EQUAL_INT(0, result.count); + TEST_ASSERT_EQUAL_INT(-1, api_detection_parse_response("{\"id\":\"x\"}", &options, &result)); + TEST_ASSERT_EQUAL_INT(-1, api_detection_parse_response("{\"detections\":\"nope\"}", &options, &result)); + TEST_ASSERT_EQUAL_INT(-1, api_detection_parse_response("garbage", &options, &result)); + TEST_ASSERT_EQUAL_INT(-1, api_detection_parse_response(NULL, &options, &result)); + TEST_ASSERT_EQUAL_INT(-1, api_detection_parse_response("{}", &options, NULL)); +} + +void test_parse_response_caps_at_max_detections(void) { + char json[8192]; + size_t len = snprintf(json, sizeof(json), "{\"detections\":["); + for (int i = 0; i < MAX_DETECTIONS + 5; i++) { + len += snprintf(json + len, sizeof(json) - len, + "%s{\"label\":\"p\",\"confidence\":0.9,\"x_min\":0,\"y_min\":0,\"x_max\":1,\"y_max\":1}", + i ? "," : ""); + TEST_ASSERT_LESS_THAN(sizeof(json) - 8, len); + } + snprintf(json + len, sizeof(json) - len, "]}"); + + api_detection_options_t options = make_options(API_DETECTION_FORMAT_LIGHT_OBJECT_DETECT, "onnx", NULL); + detection_result_t result; + TEST_ASSERT_EQUAL_INT(0, api_detection_parse_response(json, &options, &result)); + TEST_ASSERT_EQUAL_INT(MAX_DETECTIONS, result.count); +} + int main(void) { init_logger(); UNITY_BEGIN(); @@ -66,5 +366,21 @@ int main(void) { RUN_TEST(test_detect_objects_api_rejects_url_with_multiple_query_markers); RUN_TEST(test_api_detection_uses_go2rtc_snapshot_only_without_decoded_frame); RUN_TEST(test_api_detection_skips_go2rtc_snapshot_without_stream_name); + RUN_TEST(test_format_parse_accepts_known_names_and_aliases); + RUN_TEST(test_options_from_config_reads_global_settings); + RUN_TEST(test_options_from_config_falls_back_on_unknown_format); + RUN_TEST(test_options_apply_json_overrides_only_known_keys); + RUN_TEST(test_options_apply_json_rejects_bad_input); + RUN_TEST(test_build_request_url_appends_light_object_detect_params); + RUN_TEST(test_build_request_url_leaves_doods2_url_verbatim); + RUN_TEST(test_build_request_url_uses_global_config_when_options_null); + RUN_TEST(test_build_request_url_rejects_small_buffer_and_bad_args); + RUN_TEST(test_doods2_body_encodes_jpeg_threshold_and_detector); + RUN_TEST(test_doods2_body_uses_defaults_for_missing_fields); + RUN_TEST(test_parse_response_light_object_detect_flat_and_nested_boxes); + RUN_TEST(test_parse_response_doods2_maps_boxes_and_scales_confidence); + RUN_TEST(test_parse_response_doods2_error_field_fails_request); + RUN_TEST(test_parse_response_handles_null_missing_and_garbage); + RUN_TEST(test_parse_response_caps_at_max_detections); return UNITY_END(); -} \ No newline at end of file +} diff --git a/tests/unit/test_config.c b/tests/unit/test_config.c index bca17a86..6a6ba2a1 100644 --- a/tests/unit/test_config.c +++ b/tests/unit/test_config.c @@ -482,6 +482,128 @@ void test_save_config_accepts_hidden_ini_dotfile(void) { rmdir(dir); } +void test_default_config_api_detection_settings(void) { + load_default_config(&cfg); + TEST_ASSERT_EQUAL_STRING("http://localhost:8000/api/v1/detect", cfg.api_detection_url); + TEST_ASSERT_EQUAL_STRING("onnx", cfg.api_detection_backend); + TEST_ASSERT_EQUAL_STRING("light-object-detect", cfg.api_detection_format); + TEST_ASSERT_EQUAL_STRING("default", cfg.api_detection_detector_name); +} + +void test_config_set_api_detection_format_normalizes_and_rejects(void) { + load_default_config(&cfg); + + TEST_ASSERT_TRUE(config_set_api_detection_format(&cfg, "DOODS")); + TEST_ASSERT_EQUAL_STRING("doods2", cfg.api_detection_format); + + /* Unknown names are rejected and the previous value is kept. */ + TEST_ASSERT_FALSE(config_set_api_detection_format(&cfg, "deepstack")); + TEST_ASSERT_EQUAL_STRING("doods2", cfg.api_detection_format); + TEST_ASSERT_FALSE(config_set_api_detection_format(&cfg, "")); + TEST_ASSERT_FALSE(config_set_api_detection_format(NULL, "doods2")); + + TEST_ASSERT_TRUE(config_set_api_detection_format(&cfg, " Light-Object-Detect ")); + TEST_ASSERT_EQUAL_STRING("light-object-detect", cfg.api_detection_format); +} + +void test_save_config_writes_api_detection_format(void) { + char temp_dir[] = "/tmp/lightnvr_save_config_XXXXXX"; + char *dir = mkdtemp(temp_dir); + TEST_ASSERT_NOT_NULL(dir); + + char config_path[MAX_PATH_LENGTH]; + snprintf(config_path, sizeof(config_path), "%s/lightnvr.ini", dir); + + load_default_config(&cfg); + TEST_ASSERT_TRUE(config_set_api_detection_format(&cfg, "doods2")); + snprintf(cfg.api_detection_detector_name, sizeof(cfg.api_detection_detector_name), "tensorflow"); + TEST_ASSERT_EQUAL_INT(0, save_config(&cfg, config_path)); + + FILE *saved = fopen(config_path, "r"); + TEST_ASSERT_NOT_NULL(saved); + if (saved) { + char file_buffer[8192] = {0}; + size_t bytes_read = fread(file_buffer, 1, sizeof(file_buffer) - 1, saved); + TEST_ASSERT_GREATER_THAN(0, (int)bytes_read); + file_buffer[bytes_read] = '\0'; + TEST_ASSERT_NOT_NULL(strstr(file_buffer, "[api_detection]")); + TEST_ASSERT_NOT_NULL(strstr(file_buffer, "format = doods2")); + TEST_ASSERT_NOT_NULL(strstr(file_buffer, "detector_name = tensorflow")); + fclose(saved); + } + + unlink(config_path); + rmdir(dir); +} + +void test_load_config_parses_api_detection_format(void) { + char temp_dir[] = "/tmp/lightnvr_load_config_XXXXXX"; + char *dir = mkdtemp(temp_dir); + TEST_ASSERT_NOT_NULL(dir); + + char config_path[MAX_PATH_LENGTH]; + char web_root[MAX_PATH_LENGTH]; + snprintf(config_path, sizeof(config_path), "%s/test.ini", dir); + snprintf(web_root, sizeof(web_root), "%s/web", dir); + TEST_ASSERT_EQUAL_INT(0, ensure_dir(web_root)); + + FILE *config_file = fopen(config_path, "w"); + TEST_ASSERT_NOT_NULL(config_file); + fprintf(config_file, + "[general]\n" + "pid_file = %s/lightnvr.pid\n" + "log_file = %s/lightnvr.log\n\n" + "[storage]\n" + "path = %s/storage\n" + "path_hls = %s/hls\n\n" + "[models]\n" + "path = %s/models\n\n" + "[database]\n" + "path = %s/lightnvr.db\n\n" + "[web]\n" + "root = %s\n\n" + "[api_detection]\n" + "url = http://doods:8080/detect\n" + "format = DOODS2\n" + "detector_name = tensorflow\n", + dir, dir, dir, dir, dir, dir, web_root); + fclose(config_file); + + set_custom_config_path(config_path); + TEST_ASSERT_EQUAL_INT(0, load_config(&cfg)); + TEST_ASSERT_EQUAL_STRING("http://doods:8080/detect", cfg.api_detection_url); + TEST_ASSERT_EQUAL_STRING("doods2", cfg.api_detection_format); + TEST_ASSERT_EQUAL_STRING("tensorflow", cfg.api_detection_detector_name); + + /* An unknown format keeps the default rather than poisoning later requests. */ + config_file = fopen(config_path, "w"); + TEST_ASSERT_NOT_NULL(config_file); + fprintf(config_file, + "[general]\n" + "pid_file = %s/lightnvr.pid\n" + "log_file = %s/lightnvr.log\n\n" + "[storage]\n" + "path = %s/storage\n" + "path_hls = %s/hls\n\n" + "[models]\n" + "path = %s/models\n\n" + "[database]\n" + "path = %s/lightnvr.db\n\n" + "[web]\n" + "root = %s\n\n" + "[api_detection]\n" + "format = deepstack\n", + dir, dir, dir, dir, dir, dir, web_root); + fclose(config_file); + TEST_ASSERT_EQUAL_INT(0, load_config(&cfg)); + TEST_ASSERT_EQUAL_STRING("light-object-detect", cfg.api_detection_format); + TEST_ASSERT_EQUAL_STRING("default", cfg.api_detection_detector_name); + + unlink(config_path); + rmdir(web_root); + rmdir(dir); +} + void test_env_integer_whitespace_handling(void) { char temp_dir[] = "/tmp/lightnvr_load_config_XXXXXX"; char *dir = mkdtemp(temp_dir); @@ -681,6 +803,10 @@ int main(void) { RUN_TEST(test_get_loaded_config_path_initially); RUN_TEST(test_save_config_accepts_hidden_ini_dotfile); RUN_TEST(test_env_integer_whitespace_handling); + RUN_TEST(test_default_config_api_detection_settings); + RUN_TEST(test_config_set_api_detection_format_normalizes_and_rejects); + RUN_TEST(test_save_config_writes_api_detection_format); + RUN_TEST(test_load_config_parses_api_detection_format); int result = UNITY_END(); shutdown_logger(); diff --git a/web/js/components/preact/SettingsView.jsx b/web/js/components/preact/SettingsView.jsx index d5c88acb..75d1939d 100644 --- a/web/js/components/preact/SettingsView.jsx +++ b/web/js/components/preact/SettingsView.jsx @@ -120,8 +120,10 @@ export function SettingsView() { useSwap: true, swapSize: '128', detectionModelsPath: '', - apiDetectionUrl: 'http://localhost:8000/detect', + apiDetectionUrl: 'http://localhost:8000/api/v1/detect', apiDetectionBackend: 'onnx', + apiDetectionFormat: 'light-object-detect', + apiDetectionDetectorName: 'default', defaultDetectionThreshold: 50, defaultPreBuffer: 5, defaultPostBuffer: 10, @@ -363,8 +365,10 @@ export function SettingsView() { useSwap: settingsData.use_swap || false, swapSize: settingsData.swap_size?.toString() || '', detectionModelsPath: settingsData.models_path || '', - apiDetectionUrl: settingsData.api_detection_url || 'http://localhost:8000/detect', + apiDetectionUrl: settingsData.api_detection_url || 'http://localhost:8000/api/v1/detect', apiDetectionBackend: settingsData.api_detection_backend || 'onnx', + apiDetectionFormat: settingsData.api_detection_format || 'light-object-detect', + apiDetectionDetectorName: settingsData.api_detection_detector_name || 'default', defaultDetectionThreshold: settingsData.default_detection_threshold || 50, defaultPreBuffer: settingsData.pre_detection_buffer ?? 5, defaultPostBuffer: settingsData.post_detection_buffer ?? 10, @@ -503,6 +507,8 @@ export function SettingsView() { models_path: settings.detectionModelsPath, api_detection_url: settings.apiDetectionUrl, api_detection_backend: settings.apiDetectionBackend, + api_detection_format: settings.apiDetectionFormat, + api_detection_detector_name: settings.apiDetectionDetectorName, default_detection_threshold: settings.defaultDetectionThreshold, pre_detection_buffer: parseInt(settings.defaultPreBuffer, 10), post_detection_buffer: parseInt(settings.defaultPostBuffer, 10), diff --git a/web/js/components/preact/StreamConfigModal.jsx b/web/js/components/preact/StreamConfigModal.jsx index 4ee5804d..79336c5a 100644 --- a/web/js/components/preact/StreamConfigModal.jsx +++ b/web/js/components/preact/StreamConfigModal.jsx @@ -997,9 +997,6 @@ export function StreamConfigModal({

ℹī¸ {t('streamsConfig.usingDefaultApiEndpoint')}

-

- http://localhost:9001/detect -

{t('streamsConfig.configuredInLightnvrIni')} [api_detection]

diff --git a/web/js/components/preact/settings/DetectionTab.jsx b/web/js/components/preact/settings/DetectionTab.jsx index 87a8c42f..fd802a76 100644 --- a/web/js/components/preact/settings/DetectionTab.jsx +++ b/web/js/components/preact/settings/DetectionTab.jsx @@ -1,11 +1,12 @@ /** - * DetectionTab — Detection models path, API URL / backend, default detection + * DetectionTab — Detection models path, API URL / request format / backend, default detection * threshold (used when a stream enables detection-based recording). * * Part of PRD UXD_01 §5.2 / T2 settings restructure (#399). */ export function DetectionTab({ settings, handleInputChange, handleThresholdChange, canModifySettings, t }) { + const isDoods2 = settings.apiDetectionFormat === 'doods2'; return (
@@ -37,6 +38,23 @@ export function DetectionTab({ settings, handleInputChange, handleThresholdChang {t('settings.detectionModelsPathHelp')}
+
+ +
+ + {t('settings.apiDetectionFormatHelp')} +
+
@@ -48,29 +66,51 @@ export function DetectionTab({ settings, handleInputChange, handleThresholdChang value={settings.apiDetectionUrl} onChange={handleInputChange} disabled={!canModifySettings} - placeholder="http://localhost:8000/detect" + placeholder={isDoods2 ? 'http://localhost:8080/detect' : 'http://localhost:8000/api/v1/detect'} /> - {t('settings.apiDetectionUrlHelp')} + + {isDoods2 ? t('settings.apiDetectionUrlHelpDoods2') : t('settings.apiDetectionUrlHelp')} +
-
- -
- - {t('settings.apiDetectionBackendHelp')} + {!isDoods2 && ( +
+ +
+ + {t('settings.apiDetectionBackendHelp')} +
-
+ )} + {isDoods2 && ( +
+ +
+ + {t('settings.apiDetectionDetectorNameHelp')} +
+
+ )}
diff --git a/web/public/locales/en.json b/web/public/locales/en.json index 5b11c4b4..a0de76e8 100644 --- a/web/public/locales/en.json +++ b/web/public/locales/en.json @@ -1745,11 +1745,11 @@ "streamsConfig.refreshModels": "Refresh Models", "streamsConfig.usingDefaultApiEndpoint": "Using Default API Endpoint:", "streamsConfig.configuredInLightnvrIni": "Configured in lightnvr.ini under", - "streamsConfig.ensureLightObjectDetectRunning": "Make sure light-object-detect is running on this endpoint", + "streamsConfig.ensureLightObjectDetectRunning": "Make sure your detection API (light-object-detect or DOODS2, as selected in Settings \u2192 Detection) is running on this endpoint", "streamsConfig.customApiEndpointUrl": "Custom API Endpoint URL", "streamsConfig.useDefaultEndpoint": "Use Default Endpoint", - "streamsConfig.customApiEndpointPlaceholder": "http://192.168.1.100:9001/detect", - "streamsConfig.customApiEndpointHelp": "Enter the full URL to your custom detection API endpoint", + "streamsConfig.customApiEndpointPlaceholder": "http://192.168.1.100:8000/api/v1/detect", + "streamsConfig.customApiEndpointHelp": "Full URL of a detection API for this stream. Requests use the format chosen in Settings \u2192 Detection (light-object-detect or DOODS2); query parameters in the URL are passed through.", "streamsConfig.overrideWithCustomEndpoint": "Override with Custom Endpoint", "streamsConfig.detectionThreshold": "Detection Threshold", "streamsConfig.detectionThresholdHelp": "Minimum confidence level to trigger recording (0-100%)", @@ -2185,12 +2185,19 @@ "settings.detectionModelsPath": "Detection Models Path", "settings.detectionModelsPathHelp": "Directory where detection models are stored", "settings.apiDetectionUrl": "API Detection URL", - "settings.apiDetectionUrlHelp": "URL of the external object detection API endpoint", + "settings.apiDetectionUrlHelp": "light-object-detect endpoint, e.g. http://localhost:8000/api/v1/detect. The snapshot is uploaded as multipart form data with backend and confidence_threshold query parameters appended.", "settings.apiDetectionBackend": "API Detection Backend", "settings.apiDetectionBackendOnnx": "ONNX Runtime", "settings.apiDetectionBackendTflite": "TensorFlow Lite", "settings.apiDetectionBackendOpencv": "OpenCV DNN", - "settings.apiDetectionBackendHelp": "Inference backend used by the detection API", + "settings.apiDetectionBackendHelp": "Inference backend requested from light-object-detect (sent as the backend query parameter)", + "settings.apiDetectionFormat": "API Request Format", + "settings.apiDetectionFormatLightObjectDetect": "light-object-detect (multipart image upload)", + "settings.apiDetectionFormatDoods2": "DOODS2 (JSON body with base64 image)", + "settings.apiDetectionFormatHelp": "Wire format LightNVR uses when it posts snapshots to the detection API. Keep light-object-detect for light-object-detect or any server that implements its contract; choose DOODS2 for a DOODS2 server.", + "settings.apiDetectionUrlHelpDoods2": "DOODS2 /detect endpoint, e.g. http://localhost:8080/detect. The request is sent as JSON and no query parameters are added.", + "settings.apiDetectionDetectorName": "DOODS2 Detector Name", + "settings.apiDetectionDetectorNameHelp": "Sent as detector_name in each request. Must match a detector listed by the server's GET /detectors; \"default\" uses the server's default detector.", "settings.detectionEngine": "Detection Engine (Local LiteRT)", "settings.detectionEngineEnabled": "Enable Local Engine", "settings.detectionEngineEnabledHelp": "Run TFLite/LiteRT inference inside LightNVR. Streams whose model_path is a .tflite file will use this engine.",