Skip to content

v0.250.001#925

Open
paullizer wants to merge 471 commits into
mainfrom
Staging
Open

v0.250.001#925
paullizer wants to merge 471 commits into
mainfrom
Staging

Conversation

@paullizer

@paullizer paullizer commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

(v0.250.001)

New Features

  • Azure OpenAI and Model Endpoint Setup Guidance

    • Added setup guidance for legacy Azure OpenAI model discovery, runtime data-plane calls, Foundry providers, managed identity, service principal RBAC, and API-key inference limits.
    • (Ref: Azure OpenAI setup guide, model endpoint modal, Admin Settings model endpoints)

  • Tabular Semantic Kernel Large Result Pagination

  • Beta Branch Development Governance Integration

    • Prepared beta capabilities for Development governance, custom pages, user settings cache changes, agent catalog customization, file sync, workflows, workspace identities, and Data Management.
    • (Ref: Development merge resolution, governance integration)

  • User Settings Cache Optimization

    • Added request-scoped user settings memoization and a lightweight UI settings cache for Redis and no-Redis deployments.
    • (Ref: functions_settings.py, app_settings_cache.py, dark-mode.js, sidebar.js)

  • Custom Pages

    • Added administrator-managed custom pages with static assets, optional Python-backed extensions, authenticated host routes, Admin Settings controls, examples, documentation, and fail-closed coverage.
    • (Ref: route_custom_pages.py, functions_custom_pages.py)

  • Governance Controls for Endpoints, Agents, and Actions

    • Added feature policies, delegated item policies, review workflows, backend enforcement, and Admin Settings allowlists for personal, group, and global endpoints, agents, and actions.
    • (Ref: governance policies, Admin Settings Governance tab)

  • Governance and App Settings Cache Versioning

    • Added cache-version coordination so settings and governance policy changes invalidate stale worker caches while preserving hot-path performance.
    • (Ref: app settings cache, governance cache versioning)

  • Pull Request Preparation Prompt

    • Added a reusable Copilot prompt for preparing SimpleChat branches for pull requests into Development, including validation, release-note checks, and guarded push or PR creation.
    • (Ref: .github/prompts/prepare-for-pull-request.prompt.md)

  • Tableau Action

    • Added a read-only Tableau action with discovery, configuration, workspace identity support, validation, schemas, and Semantic Kernel loader integration.
    • (Ref: tableau_plugin.py, tableau_plugin_factory.py, functions_tableau_operations.py, TABLEAU_ACTION.md)

  • Snowflake Action

    • Added a read-only Snowflake action with tailored authentication, query limits, structured responses, Key Vault secret handling, governance labels, schemas, documentation, and coverage.
    • (Ref: SnowflakePlugin, SnowflakePluginFactory, SNOWFLAKE_ACTION.md)

  • Workflow Per-Document Analysis and Generated Office Exports

    • Added Analyze mode for per-document workflow execution plus SimpleChat tools for generated Word and PowerPoint exports.
    • (Ref: functions_workflow_runner.py, functions_simplechat_operations.py, WORKFLOW_PER_DOCUMENT_ANALYSIS_AND_EXPORTS.md)

  • Group Workflows

    • Added group-scoped workflow definitions, run history, per-document run items, APIs, assignment gates, admin controls, group workspace UI, and shared activity support.
    • (Ref: functions_group_workflows.py, route_backend_workflows.py, GROUP_WORKFLOWS.md)

  • Voice-Assisted Form Inputs and Agent Instruction Drafting

    • Added speech-to-text controls to supported forms and an agent Instruction Brief workflow that drafts editable Markdown instructions through the configured model.
    • (Ref: form-voice-input.js, agent_modal_stepper.js, VOICE_ASSISTED_FORM_INPUTS.md)

  • Microsoft Graph Send Mail Action

    • Microsoft Graph actions can now create drafts, prepare delayed-delivery drafts, or send mail from the signed-in user's mailbox with runtime validation and Graph scopes.
    • (Ref: MSGraphPlugin.send_mail, plugin_modal_stepper.js, MSGRAPH_SEND_MAIL_ACTION.md)

  • Workspace-Backed Chat Uploads

    • Eligible chat uploads now queue or reuse personal workspace documents as the source of truth while preserving chat attachment behavior, progress display, tags, metadata, and fallback flow.
    • (Ref: route_frontend_chats.py, route_backend_chats.py, functions_documents.py, CHAT_UPLOAD_PERSONAL_WORKSPACE_HANDOFF.md)

  • Document Intelligence PDF and Image Extraction Modes

    • Added Read/Layout/Auto extraction controls, sample-page Auto detection, extraction-mode metadata, upload source preservation, and PDF reprocess actions across workspaces.
    • (Ref: functions_content.py, functions_documents.py, DOCUMENT_INTELLIGENCE_PDF_IMAGE_EXTRACTION_MODE.md)

  • Cosmos Throughput Automation and Native Autoscale

    • Added RU monitoring, manual scale actions, background automation, per-container policies, global policy enforcement, native autoscale conversion, guarded thresholds, and deployment RBAC metadata.
    • (Ref: functions_cosmos_throughput.py, Admin Settings Scale tab, Cosmos throughput docs)

  • Workflow File Sync Triggers and Batch Resume

    • Workflows can trigger File Sync before runs, monitor sync changes, dynamically target changed synced documents, and resume failed Analyze items.
    • (Ref: functions_workflow_runner.py, functions_file_sync.py, WORKFLOW_FILE_SYNC_TRIGGERS.md)

  • Cloud Drive and File Sync Sources

    • Added OneDrive, Azure Files, SMB sync support, provider browsing, source selection, remote change-token handling, reusable identities, scheduled/manual sync, history, tags, filters, and admin source controls.
    • (Ref: functions_file_sync.py, workspace-file-sync.js, ONEDRIVE_FILE_SYNC.md, AZURE_FILES_FILE_SYNC.md, FILE_SYNC.md)

  • Global Cloud Drive Connector Identities

    • Admins can now manage File Sync cloud-drive connector credentials separately from personal user sync choices, including global connector fallback behavior.
    • (Ref: functions_workspace_identities.py, workspace-identities.js, WORKSPACE_IDENTITIES.md)

  • Conversation Feed Pagination

    • Chat startup now loads pinned, unread, and first-page recent conversations with load-more and backend title search instead of loading every accessible conversation.
    • (Ref: route_backend_conversations.py, functions_conversation_feed.py, chat-conversations.js)

  • Group File Share Approval Notifications

    • Added notifications and approval workflow for personal and group document shares, including recipient review actions and protected owner-group documents.
    • (Ref: route_backend_group_documents.py, functions_notifications.py, GROUP_FILE_SHARE_APPROVAL_NOTIFICATIONS.md)

  • Stats Time Windows and CSV Exports

    • Added 7-day, 30-day, 90-day, and custom windows plus CSV exports for personal, group, and public stats.
    • (Ref: functions_stats_windows.py, profile/group/public stats routes)

  • Personal Workflow Access Governance

    • Added personal workflow enablement, optional WorkflowUser app-role enforcement, route gating, and deployer app-role definitions.
    • (Ref: workflow access control, PERSONAL_WORKFLOWS.md, WORKFLOW_ACCESS_CONTROL_FIX.md)

  • Azure Commercial Databricks Action

    • Added Databricks SQL Statement Execution action with read-only enforcement, configuration, reusable identities, factory loading, schemas, docs, and coverage.
    • (Ref: DatabricksPlugin, DatabricksPluginFactory, DATABRICKS_ACTION_CONFIGURATION.md)

  • Model Context Protocol Actions

    • Added MCP action support with transport, authentication, timeout, tool allowlist, cached metadata, discovery, runtime invocation, and admin-only stdio restrictions.
    • (Ref: functions_mcp_operations.py, mcp_plugin.py, MCP_ACTION_CONFIGURATION.md)

  • Layered Message Masking

    • Users can layer full-message and selected-text masks independently across personal and group conversations with shared event updates.
    • (Ref: functions_message_masking.py, chat-messages.js, chat-collaboration.js)

  • Workspace and Global Identities

    • Promoted reusable identities into personal, group, public, and global identity management surfaces for File Sync, actions/plugins, model endpoints, and future integrations.
    • (Ref: functions_workspace_identities.py, route_backend_workspace_identities.py, workspace-identities.js)

  • Visio and Outlook MSG Ingestion

    • Added .vsdx and .msg ingestion, searchable Visio page chunks, citation previews, original downloads, and email-message document processing.
    • (Ref: functions_visio.py, functions_content.py, functions_documents.py, VISIO_INGESTION.md)

  • Assigned Knowledge for Agents

    • Agent creators can bind agents to governed workspace sources, documents, and tags, and chat enforces assigned search scope server-side.
    • (Ref: functions_assigned_knowledge.py, route_backend_agents.py, chat-agents.js)

  • Deep Research and Source Review Enhancements

    • Added optional Source Review, JavaScript rendering, Load More handling, model-assisted link planning, bounded deep traversal, SSRF protections, admin controls, audit details, and prompt-injection isolation.
    • (Ref: functions_source_review.py, route_backend_chats.py, SOURCE_REVIEW.md)

  • Conversation Charts and Workflow Tabular Reuse

    • Added core chat chart generation, reusable workflow tabular analysis imports, and Semantic Kernel auto tool invocation in the model-only fallback path.
    • (Ref: semantic_kernel_loader.py, functions_tabular_analysis.py, functions_workflow_runner.py)

  • Generated Markdown Artifact Viewer

    • Added View MD for generated Markdown artifacts and sanitized rendered Markdown citation previews.
    • (Ref: chat-messages.js, chat-citations.js, GENERATED_ARTIFACT_MARKDOWN_VIEW.md)

  • Tabular Related Document Evidence

    • Row-level tabular analysis can now resolve related non-tabular documents and include supporting excerpts in chat and generated structured exports.
    • (Ref: route_backend_chats.py, functions_search_service.py, TABULAR_RELATED_DOCUMENT_EVIDENCE.md)

  • Generated Artifact Workspace Promotion Approval

    • Added Add to Workspace for generated analysis artifacts with immediate personal queueing and group/public approval workflows.
    • (Ref: route_enhanced_citations.py, group/public document routes, chat-messages.js)

  • Chat Clipboard Paste Uploads

    • Users can paste images and browser-exposed files directly into chat, using the existing upload, consent, and processing path.
    • (Ref: chat-input-actions.js, CHAT_CLIPBOARD_PASTE_UPLOADS.md)

  • Staging UI Test CI/CD and Playwright Workspaces

    • Added protected Staging deployment workflow, Azure Developer CLI staging bootstrap, live UI smoke coverage, and Azure-hosted Playwright Workspace support.
    • (Ref: .github/workflows/staging-azd-ui-tests.yml, ui_tests/playwright-workspaces/)

  • Service Principal Authentication for CI UI Tests

    • Added disabled-by-default /ci-auth/session support for creating staging UI test sessions from Entra access tokens.
    • (Ref: functions_authentication.py, route_frontend_authentication.py, ENABLE_CI_BEARER_SESSION_AUTH)

  • Profile Sidebar Toggle Style Preference

    • Added a profile preference for large versus compact sidebar hide controls across full and compact sidebar templates.
    • (Ref: profile.html, sidebar templates, sidebar.css)

Bug Fixes

  • Azure OpenAI Model Discovery Identity Split

    • Legacy model discovery now uses the configured management-plane credential while inference keeps the runtime data-plane identity or key path.
    • (Ref: route_backend_models.py, deployer OpenAI RBAC, AZURE_OPENAI_MODEL_DISCOVERY_IDENTITY_FIX.md)

  • Chat Model Icon Avatars

    • Saved endpoint icons now appear on model-only assistant responses while preserving agent avatar priority.
    • (Ref: route_backend_chats.py, chat-messages.js)

  • Tabular SK Python 3.13 Compatibility

  • Python 3.12 CI and XSS Guardrail Cleanup

    • Updated CI to Python 3.12 and adjusted changed rendering paths to satisfy XSS guardrails without broad suppressions.
    • (Ref: Python 3.12 CI, Admin Settings, group workspace delete modal, profile hero image)

  • PR Readiness Guardrail Cleanup

    • Removed trailing whitespace, unnecessary |safe, UTF-8 BOM content, and documented reviewed plugin authorization boundaries for PR validation.
    • (Ref: PR readiness, XSS and BAC guardrails)

  • Governance Admin Rendering XSS Hardening

    • Rebuilt changed governance, endpoint, agent, and plugin table rendering with DOM APIs and text nodes.
    • (Ref: governance admin UI, model endpoint table, plugin table rendering)

  • Group Workflow Assignment Cleanup

    • Invalid nested JSON assignment payloads are dropped and group workflow assignment settings now preserve only valid group UUIDs.
    • (Ref: functions_settings.py, admin_settings.js, GROUP_WORKFLOW_ASSIGNMENT_CLEANUP_FIX.md)

  • Workflow Activity Navigation and Group Activity Gate

    • Workflow Activity actions now keep the current tab open and group workflow activity views use group-specific authorization instead of the personal workflow feature flag.
    • (Ref: workspace_workflows.js, workflow-activity.js, group workflow activity docs)

  • Tabular Inline Chart Handoff

    • Successful grouped CSV/XLSX results now create deterministic SimpleChat inline chart citations for streaming and non-streaming responses.
    • (Ref: route_backend_chats.py, functions_chart_operations.py, TABULAR_INLINE_CHART_HANDOFF_FIX.md)

  • Document Intelligence Upload Normalizer Import

    • Fixed extraction-mode normalizer resolution to avoid upload failures and startup circular imports.
    • (Ref: functions_content.py)

  • Cosmos Autoscale and Metrics Fixes

    • Fixed native autoscale migration, background cadence, REST metadata parsing, container metric dimensions, cached status rendering, save-button activation, refresh logging, and throughput refresh performance.
    • (Ref: functions_cosmos_throughput.py, background_tasks.py, admin_settings.js, Cosmos autoscale fix docs)

  • Advanced Conversation Search Matching

    • Advanced Search now covers legacy and collaborative stores with explicit match modes and normalized chat-type filters.
    • (Ref: route_backend_conversations.py, chat-search-modal.js, ADVANCED_CONVERSATION_SEARCH_FIX.md)

  • Visio Preview Fidelity Fixes

    • Improved Visio preview rendering for arcs, paths, master geometry, nested coordinates, connector lines, labels, embedded media, and runtime simplification.
    • (Ref: functions_visio.py, Visio preview fix docs)

  • Source Review Traversal and Citation Fixes

    • Source Review now receives full web-search citation seeds, balances seed and child pages, prioritizes relevant archive links, supports one more bounded hop, and exposes richer audit details.
    • (Ref: functions_source_review.py, route_backend_chats.py, Source Review tests/docs)

  • Authenticated Request Login Activity Tracking

    • Passive authenticated visits now contribute to login activity with throttling while preserving explicit OAuth login signals.
    • (Ref: functions_activity_logging.py, app.py, AUTHENTICATED_REQUEST_LOGIN_ACTIVITY_FIX.md)

  • Safety Violation Remediation Workflow

    • Warn, suspend, and block actions now create notifications, apply matching access restrictions, and route through approvals when required.
    • (Ref: route_backend_safety.py, functions_safety_remediation.py, admin-safety-violations.js)

  • Group and Public Workspace Hero Color Editing

    • Hero color selections now save, preview, and return through workspace branding APIs correctly.
    • (Ref: manage_group.js, manage_public_workspace.js, workspace routes)

  • Chat-Scoped Generated Tabular Exports

    • Large generated JSON and CSV exports stay attached to the active chat and are cleaned up with conversation lifecycle rules when applicable.
    • (Ref: route_backend_chats.py, functions_simplechat_operations.py, chat-messages.js)

  • Fact Memory Delete Confirmation Layering

    • Delete confirmation now opens above the Manage Fact Memories editor so users can confirm without closing the editor.
    • (Ref: profile.html, test_profile_fact_memory_editor.py)

  • Live Tabular Analysis Thought Progress

    • Long-running tabular analysis now streams tool progress into the thoughts panel while the answer is being prepared.
    • (Ref: route_backend_chats.py, chat-thoughts.js, tabular progress tests)

  • Workspace Search Document Action Gating

    • Review and Compare modes are ignored when Workspace Search is disabled, preventing stale document-selection warnings.
    • (Ref: chat-messages.js, test_chat_document_action_workspace_toggle.py)

  • Chat File Upload Client Enablement

    • Effective per-user chat upload settings now serialize to browser upload guards.
    • (Ref: chats.html, CHAT_FILE_UPLOAD_CLIENT_FLAG_FIX.md)

  • Document Auto Metadata Extraction Consistency

    • All supported upload types now run the same final automatic metadata extraction flow, including corrected public media chunk scoping.
    • (Ref: functions_documents.py, DOCUMENT_AUTO_METADATA_EXTRACTION_FIX.md)

  • Chat Stream Lifecycle Observability

    • Added backend and frontend diagnostics for active, detached, completed, errored, aborted, and recovered chat streams.
    • (Ref: route_backend_chats.py, chat-streaming.js, CHAT_STREAM_LIFECYCLE_OBSERVABILITY_FIX.md)

  • Stored XSS Hardening

    • Hardened uploaded-file previews, public tag colors, agent template actions, share/activity/masking surfaces, chat scope and conversation details, citation and file modal filenames, agent display names, workspace member rendering, Control Center public workspace members, admin rendering, and related Graph filter literals.
    • (Ref: XSS fix docs, chat-messages.js, workspace JS files, admin JS/templates, route helpers)

  • Chat Selected Document Metadata Authorization (f046)

    • Chat and tabular selected-document metadata now resolves only through authorized personal, group, and public workspace paths.
    • (Ref: route_backend_chats.py, CHAT_SELECTED_DOCUMENT_METADATA_AUTHORIZATION_FIX.md)

  • Plugin Logging Authorization Fixes

    • Recent invocation feed and destructive clear-log endpoint now require admin access while preserving unauthenticated and non-admin response contracts.
    • (Ref: route_plugin_logging.py, plugin logging authorization tests/docs)

  • Public Workspace Details Projection Hardening (f034)

    • Non-members now receive a minimal public summary instead of the full public workspace document.
    • (Ref: route_backend_public_workspaces.py, manage_public_workspace.js, PUBLIC_WORKSPACE_DETAILS_DISCLOSURE_FIX.md)

  • Approval Route Authorization Guard Consolidation (f033)

    • Admin and non-admin approval routes now share the same authorization helper before exposing approval data or executing actions.
    • (Ref: route_backend_control_center.py, functions_approvals.py)

  • Feedback Submission Ownership Enforcement (f038)

    • Feedback submissions now require conversation and message targets inside the authenticated user's own conversation before persisting data.
    • (Ref: route_backend_feedback.py, FEEDBACK_AND_PLUGIN_LOG_ACCESS_CONTROL_FIX.md)

  • Authorization State Confusion Settings Hardening

    • Public workspace selection and prompt operations now validate active workspace state server-side, and generic user-settings updates drop unsupported keys.
    • (Ref: functions_public_workspaces.py, route_backend_users.py, AUTHORIZATION_STATE_CONFUSION_SETTINGS_FIX.md)

  • Key Vault Plugin Secret Scope Enforcement (f013)

    • Plugin secret save, runtime resolution, connection tests, and cleanup now verify secret references match expected scope and source.
    • (Ref: functions_keyvault.py, semantic_kernel_loader.py, KEY_VAULT_PLUGIN_SECRET_SCOPE_ENFORCEMENT_FIX.md)

  • Log Analytics Query History User Scope Enforcement (f016)

    • Query history now binds to the authenticated user server-side and shared user-settings access denies cross-user reads/writes by default.
    • (Ref: log_analytics_plugin.py, functions_settings.py, LOG_ANALYTICS_PLUGIN_USER_SCOPE_ENFORCEMENT_FIX.md)

  • Personal Conversation Authorization Fixes

    • Hardened personal conversation deletion, message/image reads, file-content retrieval, frontend rendering, and error handling for forbidden or missing message loads.
    • (Ref: conversation routes, chat-messages.js, personal conversation authorization docs)

  • Broken Access Control IDOR Hardening

    • Enforced personal conversation ownership, canonicalized group/public scope, bound tabular blob and fact-memory access to the authorized request context, and escaped search filters.
    • (Ref: route_backend_chats.py, plugin helpers, BROKEN_ACCESS_CONTROL_IDOR_HARDENING_FIX.md)

  • Web Search Data Egress Hardening

    • Bing-grounded web search now sends only the current user message, with updated consent copy and reduced outbound invocation metadata.
    • (Ref: route_backend_chats.py, functions_settings.py, web search tests/docs)

  • Search, Groups, Approvals, and History Authorization Hardening

    • Revalidated active group selection, group prompts, approval actions, and history-grounded fallback reuse before proceeding with caller-supplied or stored identifiers.
    • (Ref: functions_search.py, functions_group.py, route hardening docs)

  • SQL ODBC Driver 18 Container Support

    • Installed and registered ODBC Driver 18 in container deployments and retried saved Driver 17 strings only when the missing-driver error applies.
    • (Ref: Dockerfile, SQL plugin files, test_sql_odbc_driver_18_support.py)

  • Entra Application Deployment Stability

    • Hardened Entra app registration scripts for MFA or conditional-access prompts and persisted app registration outputs into the selected AZD environment.
    • (Ref: Initialize-EntraApplication.ps1, Entra deployment tests)

  • Public Workspace Manage Script Syntax Fix

    • Fixed pending request handlers and delegated member-search selection initialization parser errors.
    • (Ref: manage_public_workspace.js)

  • Chat Document Dropdown Viewport Fit Fix

    • Grounded-search document dropdowns now stay inside short and mobile-influenced viewports.
    • (Ref: chat-documents.js)

User Interface Enhancements

  • Workflow Analyze Mode and Conversation Navigation

    • Added per-document Analyze switch, new-tab workflow history and alert conversation actions, and Word/PowerPoint upload toggles for SimpleChat action and agent builders.
    • (Ref: workflow UI, notifications, action and agent modals)

  • Personal Workflow Labeling

    • Renamed the existing workspace workflow surface to Personal Workflows across navigation, modal headings, primary actions, and documentation.
    • (Ref: workspace.html, workspace_workflows.js, PERSONAL_WORKFLOWS.md)

  • Custom Workspace Hero Color Swatches

    • Group and public workspace manage pages now support custom hero colors with live preview and saved custom swatch selection.
    • (Ref: manage workspace templates and JS, GROUP_PUBLIC_WORKSPACE_CUSTOM_HERO_COLORS.md)

  • Selectable Conversation-Linked Workspace Document Deletion

    • Conversation delete now lists linked workspace documents and lets users choose which to remove with the conversation.
    • (Ref: chat-conversations.js, conversation/document routes)

  • Document Intelligence Extraction Terminology and Badges

    • Renamed Read/Layout choices to Standard/Enhanced, added explanatory hover text, and moved extraction badges out of top-level document rows while preserving detail views.
    • (Ref: Admin Settings Search & Extract, workspace document views)

  • Cosmos Throughput Table Clarity and Container Policies

    • Simplified the Cosmos container table, added tooltips and setup guide test flow, preserved unavailable metrics accurately, and added a Containers modal for per-container automation policies.
    • (Ref: admin_settings.html, admin_settings.js, Cosmos throughput docs)

  • File Sync Source Configuration Flow

    • Reworked the sync source modal around selected paths, subfolders, filters, provider browsing, and OneDrive global connector guidance.
    • (Ref: workspace-file-sync.js)

  • Control Center Group Token Totals

    • Added all-time group token usage totals to the Control Center table, modal, and CSV export.
    • (Ref: route_backend_control_center.py, control-center.js, control_center.html)

  • Workspace Identity Modal Workflow

    • Replaced inline identity forms with Add, View, and Edit modals grouped around real consumers and authentication details.
    • (Ref: workspace-identities.js, functions_workspace_identities.py)

  • Deep Research Allowed-User Management Modal

    • Replaced inline user policy controls with a Manage Users modal and switched runtime behavior to allow-only access.
    • (Ref: admin_settings.html, admin_settings.js, functions_source_review.py)

  • Assistant Follow-Up Prompt Actions

    • Assistant responses can render next-step options as prompt buttons with a cancelable send countdown.
    • (Ref: chat-messages.js, chats.css)

  • Workspace Document Cards and Folder-Card Views

    • Added public workspace document cards, aligned view controls across workspaces, and improved multi-select and visible-only select-all behavior.
    • (Ref: workspace JS/CSS, workspace templates)

  • Control Center Management Pagination

    • Added page-size selectors and server-driven pagination for User, Group, and Public Workspace Management.
    • (Ref: route_backend_control_center.py, control_center.html, control-center.js)

  • Workspace Branding Heroes and Shortcuts

    • Group and public workspaces now support logo upload, active workspace hero cards, owner metadata, optional logos, and direct manage shortcuts.
    • (Ref: functions_workspace_branding.py, workspace templates and tests)

  • GitHub Pages Documentation Redesign

    • Redesigned the documentation shell with fixed top navigation, curated sidebar, mobile drawer, search, and right-side page rail.
    • (Ref: docs layouts, includes, assets, index, UI tests)

  • Chat and Sidebar Icon-Only Controls

    • Refined the chat conversation info button and compact sidebar toggle as quiet icon-only controls with accessible focus states.
    • (Ref: chats.html, _sidebar_nav.html, _sidebar_short_nav.html, sidebar.css, test_chat_sidebar_toggle_controls.py)

paullizer and others added 30 commits January 30, 2026 14:37
@paullizer
Merge branch 'main' into Staging
12910ab
@paullizer
fixed sidebar race condition (#679)
0da710a
@paullizer
Merge branch 'Staging' into Development
1e4b524
@paullizer @clarked-msft
@Bionic711 @vivche
v0.237.007 (#680)
77b3385 
* Add custom subdomain support for OpenAI and Speech Service in Terraform

- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide

* Fix Azure AI Search test connection with managed identity

Replaced REST API approach with SearchIndexClient SDK to properly handle managed identity authentication in Azure public cloud. The SDK automatically handles token acquisition and endpoint construction, eliminating the 'search_resource_manager is not defined' error that occurred with the REST API approach.

* Fix Azure AI Search test connection with managed identity

Replaced REST API approach with SearchIndexClient SDK to properly handle managed identity authentication in Azure public cloud. The SDK automatically handles token acquisition and endpoint construction, eliminating the 'search_resource_manager is not defined' error that occurred with the REST API approach.

* Corrected file folder name

* Corrected the version number to reference 0.236.012

* Removed unneeded folder and document

* Revert terraform main.tf to upstream/Development version

* updated the logging logic when running retention delete with archiving enabled (#642)

* Corrected version to 0.236.011 (#645)

* v0.237.001 (#649)

* Use Microsoft python base image

* Add python ENV vars

* Add python ENV vars

* Install deps to systme

* Add temp dir to image and pip conf support

* Add custom-ca-certificates dir

* Logo bug fix (#654)

* release note updating for github coplilot

* fixed logo bug issue

* added 2,3,4,5,6,14 days to rentention policy

* added retention policy time updates

* Rentention policy (#657)

* Critical Retention Policy Deletion Fix

* Create RETENTION_POLICY_NULL_LAST_ACTIVITY_FIX.md

* fixed retention policy runtime bug and sidebar bug (#672)

* Fix: Windows Unicode encoding issue for video uploads (#662)

- Added explicit UTF-8 encoding when reading file content on Windows
- Prevents UnicodeDecodeError when processing non-ASCII filenames
- Ensures consistent file handling across different operating systems

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md (#675)

Co-authored-by: vivche <vivche@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add custom subdomain support for OpenAI and Speech Service in Terraform (#558)

* Add custom subdomain support for OpenAI and Speech Service in Terraform

- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* 0.237.006 (#676)

* Update chat-sidebar-conversations.js

* 0.237.006

* Update release_notes.md

* fixed sidebar race condition (#679)

---------

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>
Co-authored-by: vivche <vivche@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@paullizer
Merge branch 'main' into Staging
4f296b5
Merge branch 'Development' into servicenow-integration, and moved fix…
fd66132 
… document and reference to v.0.237.008 folder
fix the version number in config.py
0c161ae
Security: Restrict group agent loading to active group only
bf90baf 
WHAT:
- Changed group agent loading from ALL user groups to ACTIVE group only
- Added security validation to prevent cross-group agent access
- Updated import from get_user_groups to require_active_group

WHY:
- Prevents users from accessing group agents outside their active group
- Fixes a security gap where selecting a group agent bypassed the active-group check
- Aligns behavior with the intended single-active-group security model

FILES:
- semantic_kernel_loader.py (approx. lines 11801280)
- GROUP_AGENT_LOADING_FIX.md

VERSION:
- 0.237.008
Fixed an instruction error that caused semantic kernel to fall back t…
8a9ad98 
…o default agent
@Bionic711
Merge pull request #640 from vivche/servicenow-integration
28a557e 
ServiceNow Integration + Group Agent/Plugin fixes
@paullizer
Fixed! The issue was caused by duplicated code blocks (#683)
82f8a89
@paullizer
Manage group frontend bug (#684)
4c24cc8 
* Fixed! The issue was caused by duplicated code blocks

* updated release notes
@paullizer
initial feature add
b66bee9
@paullizer
added tag endpoint
ef5f468
@paullizer
return unused tags too
af15c05
@eldong @paullizer
@Patrick-Davis-MSFT @nadoylemsft @cjackson202 @Bionic711 @SteveCInVA @Xeelee33 @clarked-msft @vivche @eldongormsen
Bicepfix (#690)
ef14203 
* v0.237.006 (#678)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from query

* rmv missed merge lines

* fix for AL

* upd for consistency testing

* upd abp to community

* fix copilot findings #1

* fix plotting conflict

* fix exception handling

* fix static max function invokes

* rmv unneeded decl

* rmv unneeded imports

* fix grouping dimensions

* fix abp copilot suggestions #2

* simplify methods for message reload

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

* upd pipeline to include branch name in image

* add abp to deploy

* upd instructions name/rmv abp from deploy

* fix pipeline

* mov back to Comm Cust for main inclusion

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Security/container build (#549)

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/speech managed identity (#543)

* Bugfix - deleted duplicate enable_external_healthcheck entry

* Feature - updated Speech Service to use Managed Identity in addition to the key, added MAG functionality via Azure Speech SDK since the Fast Transcription API is not available in MAG, updated Admin Setup Walkthrough so it goes to the right place in the settings when Next is clicked, updated Speech requirements in Walkthrough, rewrote Admin Configuration docs, updated/corrected Managed Identity roles in Setup Instructions Special docs.

* Update application/single_app/templates/admin_settings.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_settings.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Paul Lizer <paullizer@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

* Control center (#567)

* added group status (active, locked, upload disabled, and inactive)

Adds Azure Billing Plugin in Community Customizations

* added bulk member upload via csv for groups

* add document metadata modified activity log tracking

* activity logging for members deleted from groups

* added group activity timeline

* added notification system

* added notifications for document upload to workspaces

* fixed badge sizing

* fixed url link

* fixed badge to not show with zero notifications

* Updated notification system

* Updated approval system

* updated approval workflow

* updated notification workflow

* Fixed set active bug on my public workspace page

* Added user retention policy, updated user profile page with dashboards, retention config, and more.

* adding speed to text for chat UI

* updated the speech wave form and input field

* updated to transcribe entire recording

* fixed bug creating new conversation with auto-send

* add mic permissions

* added stream token tracking

* Added public workspace reporting

* Updated AI search sizing analysis

* added management for public workspaces

* improved public workspace management includes stats and bulk actions

* updated groups dashboard for owners and admins with stats and bulk actions

* added voice for ai to talk with users in chats

* Auto Voice Response

* for speech service, added 429 randomized response pattern to prevent thunder herding

* updated admin settings for speech services and fixed dark mode for raw log viewing

* updated video extraction card

* Added Control Center Admin and Dashboard Reader roles

* updated feedback and safety decorators so admins work unless required then those roles must be used

* Updated and Validated logic for admin roles; control center, safety, and feedback

* added support for control center admin and dashboard reader

* Development (#566)

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* updated tool tip to better inform user on status of ai response

* improve query parameters detection for swagger

* updated visual cue showing the ai is talking to the user

* moved duplicates to shared js

* replaced alert with toast.

* fixed and added log_event to exceptions

* added @user_required and improved swagger generation

* Update route_frontend_profile.py

* fixed swagger generation bug on affecting two apis

* returned keyvault to admin settings ui

* Fixed bug when running local js

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* Adding release notes

* fixed debug_debug_print

* Updated README

* Update README.md

* accepted changes

* removed files

* GitHub Actions workflow that runs Python compilation checks on all Python files in the single_app

* Upated to v0.235.002

* removed debug test file

* Updated to v0.235.003

* Update python-syntax-check.yml

* fixed disable group creation bug

* fixed bug not showing raw activity log for groups

* fixed control center access to not require admin role when enabling controlcenteradmin role

* fix documentation

* Update release_notes.md

* Update README.md

* added click restrictions to top items in control center

* bug fix - fixed group member select after search, group member removal, group member role update, and approve/reject request

* Retention execution activity log (#600)

* retention execution logging bug fix

* debug timer reset with admin save bug fix

* Create test_debug_logging_timer_preservation.py

* fixed file processing logic prevent runtime execution

* fixed bug processing execution against personal documents

* removed test logging

* Retention Policy Document Deletion Fix

* Improve execution logic for rentention policy

Now uses retention_policy_next_run timestamp - Compares current time against the stored next scheduled time. If current time >= next scheduled time, it runs.

Reduced check interval from 1 hour to 5 minutes - More responsive scheduling, ensures it catches the scheduled time promptly.

Better fallback logic - If next_run can't be parsed, falls back to checking last_run (23-hour threshold).

Runs immediately if never run before - If there's no last_run or next_run, it will execute on the first check.

* Updated branch flow

* added log_event to exceptions

* enforce-branch-flow

* v0.235.025 (#609)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from query

* rmv missed merge lines

* fix for AL

…
@paullizer
working on ui
296d987
@paullizer
Search bug fix 20260229 (#697)
a4a4224 
* fixed search in chat

* fixed buttons in manage group

* fixed issue with workspace search
@Xeelee33
Overhauled and updated file extension definition & MAG audio file tra…
541dd60 
…nscription, fixed file metadata editing (#695)

* Overhauled how file extensions are defined and referenced to reduce duplication and omission errors (mp3 was missing from list of allowed file type uploads). Added additional supported video file extensions and .heic image extension. Also rebuilt audio file transcription in MAG to use continuious recognition API instead of recognize_once()

* Adjustments to file extensions based on testing and review

* Bugfix - fixed error thrown when saving file metadata edits
@paullizer
Update release_notes.md (#698)
7c2eb0c
@paullizer
Merge branch 'Staging' into Development
e102efe
@paullizer @vivche
@clarked-msft @Bionic711 @eldong @Patrick-Davis-MSFT @nadoylemsft @cjackson202 @SteveCInVA @Xeelee33 @eldongormsen
v0.237.009 (#699)
3e208c7 
* Add custom subdomain support for OpenAI and Speech Service in Terraform

- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide

* feat: Add ServiceNow integration documentation and bug fixes

- Add comprehensive ServiceNow integration guide with OAuth 2.0 setup
- Include OpenAPI specifications for Incident Management and Knowledge Base APIs
- Add agent instructions for ServiceNow support agent
- Fix GROUP_ACTION_OAUTH_SCHEMA_MERGING: Ensure additionalFields preserved during schema merge
- Fix GROUP_AGENT_LOADING: Improve group agent loading reliability
- Fix OPENAPI_BASIC_AUTH: Support basic authentication in OpenAPI actions
- Fix AZURE_AI_SEARCH_TEST_CONNECTION: Improve AI Search connection testing
- Update version to 0.236.012

* Removed the readme files for bug fix details

* Updated servicenow integration readme

* chore: Revert custom logo changes to upstream version

* chore: Revert terraform main.tf to upstream version

* Removed the two openai sample spec downloaed from servicennow site

* Update docs/how-to/agents/ServiceNow/servicenow_agent_instructions.txt

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/open_api_specs/sample_servicenow_incident_api.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/semantic_kernel_plugins/openapi_plugin_factory.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Checked in the bug fix detail readme to docs/explanation/fixes/v0.236.012

* Added version number to the feature readme files

* Added version number to document, and removed redudant import statement

* refactor: use _ for intentionally unused variable in AI Search test

- Changed 'indexes = list(...)' to '_ = list(...)'
- Follows Python convention for discarded return values
- AI Search connection test only needs to verify the API call succeeds

* Removed azure_speech_managed_indeity_manual readme file since it is unrelated to this servicenow integration

* update version numbers to 0.236.012 in bug fix documentation

- OPENAPI_BASIC_AUTH_FIX.md: 0.235.026 → 0.236.012
- GROUP_ACTION_OAUTH_SCHEMA_MERGING_FIX.md: 0.235.028 → 0.236.012
- GROUP_AGENT_LOADING_FIX.md: 0.235.027 → 0.236.012
- AZURE_AI_SEARCH_TEST_CONNECTION_FIX.md: 0.235.004 → 0.236.012"

* Update application/single_app/semantic_kernel_loader.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/open_api_specs/sample_now_knowledge_latest_spec.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/AZURE_AI_SEARCH_TEST_CONNECTION_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/open_api_specs/sample_now_knowledge_latest_spec_basicauth.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/open_api_specs/sample_servicenow_incident_api_basicauth.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/GROUP_AGENT_LOADING_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remvoed debug statements that might include senstive info

* Rollback Azure AI Search test connection fix for separate PR

Reverted route_backend_settings.py to origin/development version and removed
AZURE_AI_SEARCH_TEST_CONNECTION_FIX.md documentation. These changes will be
submitted in a dedicated PR to keep the ServiceNow integration PR focused.

* Update application/single_app/semantic_kernel_plugins/openapi_plugin_factory.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/GROUP_AGENT_LOADING_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/GROUP_ACTION_OAUTH_SCHEMA_MERGING_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/SERVICENOW_OAUTH_SETUP.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix Azure AI Search test connection with managed identity

Replaced REST API approach with SearchIndexClient SDK to properly handle managed identity authentication in Azure public cloud. The SDK automatically handles token acquisition and endpoint construction, eliminating the 'search_resource_manager is not defined' error that occurred with the REST API approach.

* Fix Azure AI Search test connection with managed identity

Replaced REST API approach with SearchIndexClient SDK to properly handle managed identity authentication in Azure public cloud. The SDK automatically handles token acquisition and endpoint construction, eliminating the 'search_resource_manager is not defined' error that occurred with the REST API approach.

* Corrected file folder name

* Corrected the version number to reference 0.236.012

* Removed unneeded folder and document

* Revert terraform main.tf to upstream/Development version

* updated the logging logic when running retention delete with archiving enabled (#642)

* Corrected version to 0.236.011 (#645)

* v0.237.001 (#649)

* Use Microsoft python base image

* Add python ENV vars

* Add python ENV vars

* Install deps to systme

* Add temp dir to image and pip conf support

* Add custom-ca-certificates dir

* Logo bug fix (#654)

* release note updating for github coplilot

* fixed logo bug issue

* added 2,3,4,5,6,14 days to rentention policy

* added retention policy time updates

* Rentention policy (#657)

* Critical Retention Policy Deletion Fix

* Create RETENTION_POLICY_NULL_LAST_ACTIVITY_FIX.md

* Added ServiceNow support for create and publish article.  Including readme for configuration for 2 agents.

* Replace actual servicenow instance name with generic name in the readme file

* Changed version number in ServiceNow readme files to 0.237.005 since this pull request has latest changes from v0.237.004 Development branch

* Enhance ServiceNow agent for managing new KB article creation

* Added readme and open ai specs and agent instructions to support ServiceNow asset management

* Remove any references to actual ServiceNow instances

* fixed retention policy runtime bug and sidebar bug (#672)

* Fix: Windows Unicode encoding issue for video uploads (#662)

- Added explicit UTF-8 encoding when reading file content on Windows
- Prevents UnicodeDecodeError when processing non-ASCII filenames
- Ensures consistent file handling across different operating systems

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md (#675)

Co-authored-by: vivche <vivche@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add custom subdomain support for OpenAI and Speech Service in Terraform (#558)

* Add custom subdomain support for OpenAI and Speech Service in Terraform

- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* 0.237.006 (#676)

* Update chat-sidebar-conversations.js

* 0.237.006

* docs: Update release notes for ServiceNow integration and bug fixes

- Add comprehensive ServiceNow integration documentation entry
- Document 4 critical bug fixes:
  - OpenAPI Basic Authentication credential parsing fix
  - Group Action OAuth schema merging fix
  - Group Agent loading in per-user kernel mode fix
  - Azure AI Search test connection fix
- Move debug_print import to top-level in openapi_plugin_factory.py

Version: 0.237.005

* Update release_notes.md

* resolve conflict

* fixed sidebar race condition (#679)

* fix the version number in config.py

* Security: Restrict group agent loading to active group only

WHAT:
- Changed group agent loading from ALL user groups to ACTIVE group only
- Added security validation to prevent cross-group agent access
- Updated import from get_user_groups to require_active_group

WHY:
- Prevents users from accessing group agents outside their active group
- Fixes a security gap where selecting a group agent bypassed the active-group check
- Aligns behavior with the intended single-active-group security model

FILES:
- semantic_kernel_loader.py (approx. lines 11801280)
- GROUP_AGENT_LOADING_FIX.md

VERSION:
- 0.237.008

* Fixed an instruction error that caused semantic kernel to fall back to default agent

* Fixed! The issue was caused by duplicated code blocks (#683)

* Manage group frontend bug (#684)

* Fixed! The issue was caused by duplicated code blocks

* updated release notes

* Bicepfix (#690)

* v0.237.006 (#678)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from query

* rmv missed merge lines

* fix for AL

* upd for consistency testing

* upd abp to community

* fix copilot findings #1

* fix plotting conflict

* fix exception handling

* fix static max function invokes

* rmv unneeded decl

* rmv unneeded imports

* fix grouping dimensions

* fix abp copilot suggestions #2

* simplify methods for message reload

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

* upd pipeline to include branch name in image

* add abp to deploy

* upd instructions name/rmv abp from deploy

* fix pipeline

* mov back to Comm Cust for main inclusion

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Security/container build (#549)

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/speech managed identity (#543)

* Bugfix - deleted duplicate enable_external_healthcheck entry

* Feature - updated Speech Service to use Managed Identity in addition to the key, added MAG functionality via Azure Speech SDK since the Fast Transcription API is not available in MAG, updated Admin Setup Walkthrough so it goes to the right place in the settings when Next is clicked, updated Speech requirements in Walkthrough, rewrote Admin Configuration docs, updated/corrected Managed Identity roles in Setup Instructions Special docs.

* Update application/single_app/templates/admin_settings.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_settings.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Paul Lizer <paullizer@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

* Control center (#567)

* added group status (active, locked, upload disabled, and inactive)

Adds Azure Billing Plugin in Community Customizations

* added bulk member upload via csv for groups

* add document metadata modified activity log tracking

* activity logging for members deleted from groups

* added group activity timeline

* added notification system

* added notifications for document upload to workspaces

* fixed badge sizing

* fixed url link

* fixed badge to not show with zero notifications

* Updated notification system

* Updated approval system

* updated approval workflow

* updated notification workflow

* Fixed set active bug on my public workspace page

* Added user retention policy, updated user profile page with dashboards, retention config, and more.

* adding speed to text for chat UI

* updated the speech wave form and input field

* updated to transcribe entire recording

* fixed bug creating new conversation with auto-send

* add mic permissions

* added stream token tracking

* Added public workspace reporting

* Updated AI search sizing analysis

* added management for public workspaces

* improved public workspace management includes stats and bulk actions

* updated groups dashboard for owners and admins with stats and bulk actions

* added voice for ai to talk with users in chats

* Auto Voice Response

* for speech service, added 429 randomized response pattern to prevent thunder herding

* updated admin settings for speech services and fixed dark mode for raw log viewing

* updated video extraction card

* Added Control Center Admin and Dashboard Reader roles

* updated feedback and safety decorators so admins work unless required then those roles must be used

* Updated and Validated logic for admin roles; control center, safety, and feedback

* added support for control center admin and dashboard reader

* Development (#566)

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* updated tool tip to better inform user on status of ai response

* improve query parameters detection for swagger

* updated visual cue showing the ai is talking to the user

* moved duplicates to shared js

* replaced alert with toast.

* fixed and added log_event to exceptions

* added @user_required and improved swagger generation

* Update route_frontend_profile.py

* fixed swagger generation bug on affecting two apis

* returned keyvault to admin settings ui

* Fixed bug when running local js

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* Adding release notes

* fixed debug_debug_print

* Updated README

* Update README.md

* accepted changes

* removed files

* GitHub Actions workflow that runs Python compilation checks on all Python files in the single_app

* Upated to v0.235.002

* removed debug test file

* Updated to v0.235.003

* Update python-syntax-check.yml

* fixed disable group creation bug

* fixed bug not showing raw activity log for groups

* fixed control center access to not require admin role when enabling controlcenteradmin role

* fix documentation

* Update release_notes.md

* Update README.md

* added click restrictions to top items in control center

* bug fix - fixed group member select after search, group member removal, group member role update, and approve/reject request

* Retention execution activity log (#600)

* retention execution logging bug fix

* debug timer reset with admin save bug fix

* Create test_debug_logging_timer_preservation.py

* fixed file processing logic prevent runtime execution

* fixed bug processing execution against personal documents

* removed test logging

* Retention Policy Document Deletion Fix

* Improve execution logic for rentention policy

Now uses retention_policy_next_run timestamp - Compares current time against the stored next scheduled time. If current time >= next scheduled time, it runs.

Reduced check interval from 1 hour to 5 minutes - More responsive scheduling, ensures it catches the scheduled time promptly.

Better fallback logic - If next_run can't be parsed, falls back to checking last_run (23-hour threshold).

Runs immediately if never run before - If there's no last_run or next_run, it will execute on the first check.

* Updated branch flow

* added log_event to exceptions

* enforce-branch-flow

* v0.235.025 (#609)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion…
@paullizer
Merge branch 'main' into Staging
6862f6e
@paullizer
removed duplicate code causing bugs (#701)
2429489
@paullizer
Chat file upload error (#709)
b529d60 
* improved file support analysis method and resolved bug

* Update release_notes.md
@paullizer
Merge branch 'Staging' into Development
0cd14b9
@paullizer @vivche
@clarked-msft @Bionic711 @eldong @Patrick-Davis-MSFT @nadoylemsft @cjackson202 @SteveCInVA @Xeelee33 @eldongormsen
Development (#710)
19b37bd 
* Add custom subdomain support for OpenAI and Speech Service in Terraform

- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide

* feat: Add ServiceNow integration documentation and bug fixes

- Add comprehensive ServiceNow integration guide with OAuth 2.0 setup
- Include OpenAPI specifications for Incident Management and Knowledge Base APIs
- Add agent instructions for ServiceNow support agent
- Fix GROUP_ACTION_OAUTH_SCHEMA_MERGING: Ensure additionalFields preserved during schema merge
- Fix GROUP_AGENT_LOADING: Improve group agent loading reliability
- Fix OPENAPI_BASIC_AUTH: Support basic authentication in OpenAPI actions
- Fix AZURE_AI_SEARCH_TEST_CONNECTION: Improve AI Search connection testing
- Update version to 0.236.012

* Removed the readme files for bug fix details

* Updated servicenow integration readme

* chore: Revert custom logo changes to upstream version

* chore: Revert terraform main.tf to upstream version

* Removed the two openai sample spec downloaed from servicennow site

* Update docs/how-to/agents/ServiceNow/servicenow_agent_instructions.txt

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/open_api_specs/sample_servicenow_incident_api.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/semantic_kernel_plugins/openapi_plugin_factory.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Checked in the bug fix detail readme to docs/explanation/fixes/v0.236.012

* Added version number to the feature readme files

* Added version number to document, and removed redudant import statement

* refactor: use _ for intentionally unused variable in AI Search test

- Changed 'indexes = list(...)' to '_ = list(...)'
- Follows Python convention for discarded return values
- AI Search connection test only needs to verify the API call succeeds

* Removed azure_speech_managed_indeity_manual readme file since it is unrelated to this servicenow integration

* update version numbers to 0.236.012 in bug fix documentation

- OPENAPI_BASIC_AUTH_FIX.md: 0.235.026 → 0.236.012
- GROUP_ACTION_OAUTH_SCHEMA_MERGING_FIX.md: 0.235.028 → 0.236.012
- GROUP_AGENT_LOADING_FIX.md: 0.235.027 → 0.236.012
- AZURE_AI_SEARCH_TEST_CONNECTION_FIX.md: 0.235.004 → 0.236.012"

* Update application/single_app/semantic_kernel_loader.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/open_api_specs/sample_now_knowledge_latest_spec.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/AZURE_AI_SEARCH_TEST_CONNECTION_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/open_api_specs/sample_now_knowledge_latest_spec_basicauth.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/open_api_specs/sample_servicenow_incident_api_basicauth.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/GROUP_AGENT_LOADING_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remvoed debug statements that might include senstive info

* Rollback Azure AI Search test connection fix for separate PR

Reverted route_backend_settings.py to origin/development version and removed
AZURE_AI_SEARCH_TEST_CONNECTION_FIX.md documentation. These changes will be
submitted in a dedicated PR to keep the ServiceNow integration PR focused.

* Update application/single_app/semantic_kernel_plugins/openapi_plugin_factory.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/GROUP_AGENT_LOADING_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/GROUP_ACTION_OAUTH_SCHEMA_MERGING_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/SERVICENOW_OAUTH_SETUP.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix Azure AI Search test connection with managed identity

Replaced REST API approach with SearchIndexClient SDK to properly handle managed identity authentication in Azure public cloud. The SDK automatically handles token acquisition and endpoint construction, eliminating the 'search_resource_manager is not defined' error that occurred with the REST API approach.

* Fix Azure AI Search test connection with managed identity

Replaced REST API approach with SearchIndexClient SDK to properly handle managed identity authentication in Azure public cloud. The SDK automatically handles token acquisition and endpoint construction, eliminating the 'search_resource_manager is not defined' error that occurred with the REST API approach.

* Corrected file folder name

* Corrected the version number to reference 0.236.012

* Removed unneeded folder and document

* Revert terraform main.tf to upstream/Development version

* updated the logging logic when running retention delete with archiving enabled (#642)

* Corrected version to 0.236.011 (#645)

* v0.237.001 (#649)

* Use Microsoft python base image

* Add python ENV vars

* Add python ENV vars

* Install deps to systme

* Add temp dir to image and pip conf support

* Add custom-ca-certificates dir

* Logo bug fix (#654)

* release note updating for github coplilot

* fixed logo bug issue

* added 2,3,4,5,6,14 days to rentention policy

* added retention policy time updates

* Rentention policy (#657)

* Critical Retention Policy Deletion Fix

* Create RETENTION_POLICY_NULL_LAST_ACTIVITY_FIX.md

* Added ServiceNow support for create and publish article.  Including readme for configuration for 2 agents.

* Replace actual servicenow instance name with generic name in the readme file

* Changed version number in ServiceNow readme files to 0.237.005 since this pull request has latest changes from v0.237.004 Development branch

* Enhance ServiceNow agent for managing new KB article creation

* Added readme and open ai specs and agent instructions to support ServiceNow asset management

* Remove any references to actual ServiceNow instances

* fixed retention policy runtime bug and sidebar bug (#672)

* Fix: Windows Unicode encoding issue for video uploads (#662)

- Added explicit UTF-8 encoding when reading file content on Windows
- Prevents UnicodeDecodeError when processing non-ASCII filenames
- Ensures consistent file handling across different operating systems

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md (#675)

Co-authored-by: vivche <vivche@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add custom subdomain support for OpenAI and Speech Service in Terraform (#558)

* Add custom subdomain support for OpenAI and Speech Service in Terraform

- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* 0.237.006 (#676)

* Update chat-sidebar-conversations.js

* 0.237.006

* docs: Update release notes for ServiceNow integration and bug fixes

- Add comprehensive ServiceNow integration documentation entry
- Document 4 critical bug fixes:
  - OpenAPI Basic Authentication credential parsing fix
  - Group Action OAuth schema merging fix
  - Group Agent loading in per-user kernel mode fix
  - Azure AI Search test connection fix
- Move debug_print import to top-level in openapi_plugin_factory.py

Version: 0.237.005

* Update release_notes.md

* resolve conflict

* fixed sidebar race condition (#679)

* fix the version number in config.py

* Security: Restrict group agent loading to active group only

WHAT:
- Changed group agent loading from ALL user groups to ACTIVE group only
- Added security validation to prevent cross-group agent access
- Updated import from get_user_groups to require_active_group

WHY:
- Prevents users from accessing group agents outside their active group
- Fixes a security gap where selecting a group agent bypassed the active-group check
- Aligns behavior with the intended single-active-group security model

FILES:
- semantic_kernel_loader.py (approx. lines 11801280)
- GROUP_AGENT_LOADING_FIX.md

VERSION:
- 0.237.008

* Fixed an instruction error that caused semantic kernel to fall back to default agent

* Fixed! The issue was caused by duplicated code blocks (#683)

* Manage group frontend bug (#684)

* Fixed! The issue was caused by duplicated code blocks

* updated release notes

* Bicepfix (#690)

* v0.237.006 (#678)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from query

* rmv missed merge lines

* fix for AL

* upd for consistency testing

* upd abp to community

* fix copilot findings #1

* fix plotting conflict

* fix exception handling

* fix static max function invokes

* rmv unneeded decl

* rmv unneeded imports

* fix grouping dimensions

* fix abp copilot suggestions #2

* simplify methods for message reload

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

* upd pipeline to include branch name in image

* add abp to deploy

* upd instructions name/rmv abp from deploy

* fix pipeline

* mov back to Comm Cust for main inclusion

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Security/container build (#549)

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/speech managed identity (#543)

* Bugfix - deleted duplicate enable_external_healthcheck entry

* Feature - updated Speech Service to use Managed Identity in addition to the key, added MAG functionality via Azure Speech SDK since the Fast Transcription API is not available in MAG, updated Admin Setup Walkthrough so it goes to the right place in the settings when Next is clicked, updated Speech requirements in Walkthrough, rewrote Admin Configuration docs, updated/corrected Managed Identity roles in Setup Instructions Special docs.

* Update application/single_app/templates/admin_settings.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_settings.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Paul Lizer <paullizer@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

* Control center (#567)

* added group status (active, locked, upload disabled, and inactive)

Adds Azure Billing Plugin in Community Customizations

* added bulk member upload via csv for groups

* add document metadata modified activity log tracking

* activity logging for members deleted from groups

* added group activity timeline

* added notification system

* added notifications for document upload to workspaces

* fixed badge sizing

* fixed url link

* fixed badge to not show with zero notifications

* Updated notification system

* Updated approval system

* updated approval workflow

* updated notification workflow

* Fixed set active bug on my public workspace page

* Added user retention policy, updated user profile page with dashboards, retention config, and more.

* adding speed to text for chat UI

* updated the speech wave form and input field

* updated to transcribe entire recording

* fixed bug creating new conversation with auto-send

* add mic permissions

* added stream token tracking

* Added public workspace reporting

* Updated AI search sizing analysis

* added management for public workspaces

* improved public workspace management includes stats and bulk actions

* updated groups dashboard for owners and admins with stats and bulk actions

* added voice for ai to talk with users in chats

* Auto Voice Response

* for speech service, added 429 randomized response pattern to prevent thunder herding

* updated admin settings for speech services and fixed dark mode for raw log viewing

* updated video extraction card

* Added Control Center Admin and Dashboard Reader roles

* updated feedback and safety decorators so admins work unless required then those roles must be used

* Updated and Validated logic for admin roles; control center, safety, and feedback

* added support for control center admin and dashboard reader

* Development (#566)

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* updated tool tip to better inform user on status of ai response

* improve query parameters detection for swagger

* updated visual cue showing the ai is talking to the user

* moved duplicates to shared js

* replaced alert with toast.

* fixed and added log_event to exceptions

* added @user_required and improved swagger generation

* Update route_frontend_profile.py

* fixed swagger generation bug on affecting two apis

* returned keyvault to admin settings ui

* Fixed bug when running local js

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* Adding release notes

* fixed debug_debug_print

* Updated README

* Update README.md

* accepted changes

* removed files

* GitHub Actions workflow that runs Python compilation checks on all Python files in the single_app

* Upated to v0.235.002

* removed debug test file

* Updated to v0.235.003

* Update python-syntax-check.yml

* fixed disable group creation bug

* fixed bug not showing raw activity log for groups

* fixed control center access to not require admin role when enabling controlcenteradmin role

* fix documentation

* Update release_notes.md

* Update README.md

* added click restrictions to top items in control center

* bug fix - fixed group member select after search, group member removal, group member role update, and approve/reject request

* Retention execution activity log (#600)

* retention execution logging bug fix

* debug timer reset with admin save bug fix

* Create test_debug_logging_timer_preservation.py

* fixed file processing logic prevent runtime execution

* fixed bug processing execution against personal documents

* removed test logging

* Retention Policy Document Deletion Fix

* Improve execution logic for rentention policy

Now uses retention_policy_next_run timestamp - Compares current time against the stored next scheduled time. If current time >= next scheduled time, it runs.

Reduced check interval from 1 hour to 5 minutes - More responsive scheduling, ensures it catches the scheduled time promptly.

Better fallback logic - If next_run can't be parsed, falls back to checking last_run (23-hour threshold).

Runs immediately if never run before - If there's no last_run or next_run, it will execute on the first check.

* Updated branch flow

* added log_event to exceptions

* enforce-branch-flow

* v0.235.025 (#609)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustio…
@paullizer
Merge branch 'main' into Staging
98e86b6
@paullizer
edit folders
7381359
@paullizer
sorting fixed
66f9287
Bionic711 and others added 19 commits May 7, 2026 17:27
@Bionic711 @nadoylemsft
Bugfix/241.006 (#885)
58f017d 
* fix for hardocded cognitive services scope

* fix for fact memory display and display:none

* fix for enhanced citations, pdf, images,

* upd adding limited mobile support

support added for small portrait screens
covers personal/group workspaces, chat, profile, notifications

* upd mobile UI

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>
@paullizer
Update release notes for version v0.241.022
acb545e 
Updated release notes for version v0.241.022 with new features and bug fixes related to XSS hardening, authorization enforcement, and UI improvements.
@paullizer
Fix formatting and enhance mobile UI support description
9ab4f52 
Correct formatting and improve clarity in release notes.
@paullizer
Update version number in config.py
11f00a1
@paullizer
Change release version to v0.241.007
570bb91 
Updated release notes to reflect version v0.241.007 and added new feature details.
@paullizer @vivche
@clarked-msft @Bionic711 @eldong @Patrick-Davis-MSFT @nadoylemsft @cjackson202 @SteveCInVA @Xeelee33 @eldongormsen @chadpalmer
v0.241.007 (#886)
0f9b4d2 
* Remvoed debug statements that might include senstive info

* Rollback Azure AI Search test connection fix for separate PR

Reverted route_backend_settings.py to origin/development version and removed
AZURE_AI_SEARCH_TEST_CONNECTION_FIX.md documentation. These changes will be
submitted in a dedicated PR to keep the ServiceNow integration PR focused.

* Update application/single_app/semantic_kernel_plugins/openapi_plugin_factory.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/GROUP_AGENT_LOADING_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/explanation/fixes/v0.236.012/GROUP_ACTION_OAUTH_SCHEMA_MERGING_FIX.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/how-to/agents/ServiceNow/SERVICENOW_OAUTH_SETUP.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix Azure AI Search test connection with managed identity

Replaced REST API approach with SearchIndexClient SDK to properly handle managed identity authentication in Azure public cloud. The SDK automatically handles token acquisition and endpoint construction, eliminating the 'search_resource_manager is not defined' error that occurred with the REST API approach.

* Fix Azure AI Search test connection with managed identity

Replaced REST API approach with SearchIndexClient SDK to properly handle managed identity authentication in Azure public cloud. The SDK automatically handles token acquisition and endpoint construction, eliminating the 'search_resource_manager is not defined' error that occurred with the REST API approach.

* Corrected file folder name

* Corrected the version number to reference 0.236.012

* Removed unneeded folder and document

* Revert terraform main.tf to upstream/Development version

* updated the logging logic when running retention delete with archiving enabled (#642)

* Corrected version to 0.236.011 (#645)

* v0.237.001 (#649)

* Use Microsoft python base image

* Add python ENV vars

* Add python ENV vars

* Install deps to systme

* Add temp dir to image and pip conf support

* Add custom-ca-certificates dir

* Logo bug fix (#654)

* release note updating for github coplilot

* fixed logo bug issue

* added 2,3,4,5,6,14 days to rentention policy

* added retention policy time updates

* Rentention policy (#657)

* Critical Retention Policy Deletion Fix

* Create RETENTION_POLICY_NULL_LAST_ACTIVITY_FIX.md

* Added ServiceNow support for create and publish article.  Including readme for configuration for 2 agents.

* Replace actual servicenow instance name with generic name in the readme file

* Changed version number in ServiceNow readme files to 0.237.005 since this pull request has latest changes from v0.237.004 Development branch

* Enhance ServiceNow agent for managing new KB article creation

* Added readme and open ai specs and agent instructions to support ServiceNow asset management

* Remove any references to actual ServiceNow instances

* fixed retention policy runtime bug and sidebar bug (#672)

* Fix: Windows Unicode encoding issue for video uploads (#662)

- Added explicit UTF-8 encoding when reading file content on Windows
- Prevents UnicodeDecodeError when processing non-ASCII filenames
- Ensures consistent file handling across different operating systems

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md (#675)

Co-authored-by: vivche <vivche@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add custom subdomain support for OpenAI and Speech Service in Terraform (#558)

* Add custom subdomain support for OpenAI and Speech Service in Terraform

- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* 0.237.006 (#676)

* Update chat-sidebar-conversations.js

* 0.237.006

* docs: Update release notes for ServiceNow integration and bug fixes

- Add comprehensive ServiceNow integration documentation entry
- Document 4 critical bug fixes:
  - OpenAPI Basic Authentication credential parsing fix
  - Group Action OAuth schema merging fix
  - Group Agent loading in per-user kernel mode fix
  - Azure AI Search test connection fix
- Move debug_print import to top-level in openapi_plugin_factory.py

Version: 0.237.005

* Update release_notes.md

* resolve conflict

* fixed sidebar race condition (#679)

* fix the version number in config.py

* Security: Restrict group agent loading to active group only

WHAT:
- Changed group agent loading from ALL user groups to ACTIVE group only
- Added security validation to prevent cross-group agent access
- Updated import from get_user_groups to require_active_group

WHY:
- Prevents users from accessing group agents outside their active group
- Fixes a security gap where selecting a group agent bypassed the active-group check
- Aligns behavior with the intended single-active-group security model

FILES:
- semantic_kernel_loader.py (approx. lines 11801280)
- GROUP_AGENT_LOADING_FIX.md

VERSION:
- 0.237.008

* Fixed an instruction error that caused semantic kernel to fall back to default agent

* Fixed! The issue was caused by duplicated code blocks (#683)

* Manage group frontend bug (#684)

* Fixed! The issue was caused by duplicated code blocks

* updated release notes

* initial feature add

* added tag endpoint

* return unused tags too

* Bicepfix (#690)

* v0.237.006 (#678)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from query

* rmv missed merge lines

* fix for AL

* upd for consistency testing

* upd abp to community

* fix copilot findings #1

* fix plotting conflict

* fix exception handling

* fix static max function invokes

* rmv unneeded decl

* rmv unneeded imports

* fix grouping dimensions

* fix abp copilot suggestions #2

* simplify methods for message reload

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

* upd pipeline to include branch name in image

* add abp to deploy

* upd instructions name/rmv abp from deploy

* fix pipeline

* mov back to Comm Cust for main inclusion

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Security/container build (#549)

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/speech managed identity (#543)

* Bugfix - deleted duplicate enable_external_healthcheck entry

* Feature - updated Speech Service to use Managed Identity in addition to the key, added MAG functionality via Azure Speech SDK since the Fast Transcription API is not available in MAG, updated Admin Setup Walkthrough so it goes to the right place in the settings when Next is clicked, updated Speech requirements in Walkthrough, rewrote Admin Configuration docs, updated/corrected Managed Identity roles in Setup Instructions Special docs.

* Update application/single_app/templates/admin_settings.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_settings.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Paul Lizer <paullizer@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

* Control center (#567)

* added group status (active, locked, upload disabled, and inactive)

Adds Azure Billing Plugin in Community Customizations

* added bulk member upload via csv for groups

* add document metadata modified activity log tracking

* activity logging for members deleted from groups

* added group activity timeline

* added notification system

* added notifications for document upload to workspaces

* fixed badge sizing

* fixed url link

* fixed badge to not show with zero notifications

* Updated notification system

* Updated approval system

* updated approval workflow

* updated notification workflow

* Fixed set active bug on my public workspace page

* Added user retention policy, updated user profile page with dashboards, retention config, and more.

* adding speed to text for chat UI

* updated the speech wave form and input field

* updated to transcribe entire recording

* fixed bug creating new conversation with auto-send

* add mic permissions

* added stream token tracking

* Added public workspace reporting

* Updated AI search sizing analysis

* added management for public workspaces

* improved public workspace management includes stats and bulk actions

* updated groups dashboard for owners and admins with stats and bulk actions

* added voice for ai to talk with users in chats

* Auto Voice Response

* for speech service, added 429 randomized response pattern to prevent thunder herding

* updated admin settings for speech services and fixed dark mode for raw log viewing

* updated video extraction card

* Added Control Center Admin and Dashboard Reader roles

* updated feedback and safety decorators so admins work unless required then those roles must be used

* Updated and Validated logic for admin roles; control center, safety, and feedback

* added support for control center admin and dashboard reader

* Development (#566)

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* updated tool tip to better inform user on status of ai response

* improve query parameters detection for swagger

* updated visual cue showing the ai is talking to the user

* moved duplicates to shared js

* replaced alert with toast.

* fixed and added log_event to exceptions

* added @user_required and improved swagger generation

* Update route_frontend_profile.py

* fixed swagger generation bug on affecting two apis

* returned keyvault to admin settings ui

* Fixed bug when running local js

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* Adding release notes

* fixed debug_debug_print

* Updated README

* Update README.md

* accepted changes

* removed files

* GitHub Actions workflow that runs Python compilation checks on all Python files in the single_app

* Upated to v0.235.002

* removed debug test file

* Updated to v0.235.003

* Update python-syntax-check.yml

* fixed disable group creation bug

* fixed bug not showing raw activity log for groups

* fixed control center access to not require admin role when enabling controlcenteradmin role

* fix documentation

* Update release_notes.md

* Update README.md

* added click restrictions to top items in control center

* bug fix - fixed group member select after search, group member removal, group member role update, and approve/reject request

* Retention execution activity log (#600)

* retention execution logging bug fix

* debug timer reset with admin save bug fix

* Create test_debug_logging_timer_preservation.py

* fixed file processing logic prevent runtime execution

* fixed bug processing execution against personal documents

* removed test logging

* Retention Policy Document Deletion Fix

* Improve execution logic for rentention policy

Now uses retention_policy_next_run timestamp - Compares current time against the stored next scheduled time. If current time >= next scheduled time, it runs.

Reduced check interval from 1 hour to 5 minutes - More responsive scheduling, ensures it catches the scheduled time promptly.

Better fallback logic - If next_run can't be parsed, falls back to checking last_run (23-hour threshold).

Runs immediately if never run before - If there's no last_run or next_run, it will execute on the first check.

* Updated branch flow

* added log_event to exceptions

* enforce-branch-flow

* v0.235.025 (#609)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and messa…
@Bionic711 @nadoylemsft
Feature/governance (#904)
c1a0637 
* init governance

* cont of governance

* upd PR prompt

* fix security findings/init prepare run

* upd gitignore

* add custom pages with default access

* fix for metadata extraction model selection for multiendpoint

* fix for metadata extraction and nav

* fix xss

* fix xss

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>
@Bionic711 @nadoylemsft
Feature/governance (#908)
3781c1c 
* init governance

* cont of governance

* upd PR prompt

* fix security findings/init prepare run

* upd gitignore

* add custom pages with default access

* fix for metadata extraction model selection for multiendpoint

* fix for metadata extraction and nav

* fix xss

* fix xss

* init user settings cache improvements

* Prepare user settings cache PR

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>
@paullizer
Beta branch integration and PR readiness (#912)
3fab884 
* search api and search plugin

* initial

* initial

* +1

* Implemented the collaboration shared AI workflow end to end

* cosmo plugin, simple chat plugin, search plugin, image gen fix, mulit-user source fix

* updated msgraph, muilt-user image gen, created workflows, created search plugin, simple chat plugin

* realtime workflow activity log

* updated workflow UI

* updated ui for chat

* updated message visuals for maps

* inline images

* chat image update

* updated maps

* sql fix

* updated sk tool calls

* Added video support in ai message

* added exhaustive review with progress bar

* doc compare

* conversation and export summary

* removed

* bug fixes

* fixed odbc container issue

* awk fix

* updated to fix odbc pathing

* Delete logs.txt

* deployer versioning

* fix  global agent bug (#877)

global agent was  being classified as personal which meant that the user agent needed to be enabled for global agents to work too.

* improve multi-select, and customize doc action scale

* added reconnect for doc actions and debug logging

* auth retry fixes

* bug fix for upload, create group, change group, activity logging for doc actions

* export to powerpoint

* auth hardening and xss hardening updates (#881)

* auth hardening and xss hardening updates

* Update manage_group.js

* added paste into chat for images and files, improve streaming lifecycle performance, bug fix for workspace selection

* fix workspace selection bug

* added count to tagging

* updated profile, feedback, and violations pages

* improved review, compare, added tabular support to those actions.

* tabular analysis improvements

* Ability for tabular analysis to search for files referenced in the tabular file

* table csv bug fix

* moved group workspace hero

* fix toast bug, add public workspace search

* Update test_chat_collaboration_invite_toast_button.py

* Adds the v0.241.008 release bundle covering staging CI/CD, deployment setup, UI polish, docs, and stability fixes.

Highlights:
- Add Staging branch GitHub Actions workflow with Azure OIDC auth,
  azd deployment, warm-up handling, and UI smoke tests.
- Add Microsoft Playwright Workspaces runner plus Python and Node
  staging chat smoke coverage.
- Add disabled-by-default CI bearer-session auth so GitHub Actions can
  create a test session from a fresh Entra service principal token.
- Improve Entra app bootstrap with Microsoft Graph MFA recovery and
  AZD environment persistence for app registration values.
- Fix SQL Server and Azure SQL ODBC Driver 18 container support,
  runtime registration, and legacy Driver 17 fallback behavior.
- Fix public workspace manage script parsing and chat document dropdown
  viewport placement.
- Add profile sidebar toggle style preference and refine chat/sidebar
  icon-only controls.
- Redesign GitHub Pages documentation shell and update v0.241.008
  release notes, feature docs, fix docs, and regression coverage.

* improve UX for compare, my group/pub workspace in profile, control center auto refresh & pagination, activity log auto refresh

Highlights:
- Adds Profile workspace tabs, Control Center scheduled refresh, Activity Logs auto-refresh, management pagination, and persistent sidebar state.
- Expands chat and workspace flows with document analysis updates, generated Markdown previews, generated artifact promotion, tabular related-document evidence, clipboard paste uploads, and chat-scoped generated exports.
- Improves document actions/comparison, structured export handling, live progress thoughts, citation rendering, per-message export behavior, and conversation export output.
- Hardens authorization and stored-XSS boundaries across selected documents, public workspaces, approvals, feedback, plugin logs, Key Vault plugin secrets, settings-backed scope state, chat modals, uploaded-file previews, tags, share flows, and admin/member surfaces.
- Updates config versioning, release notes, feature/fix documentation, functional tests, and UI regression coverage for v0.241.028.

* Review to analyze, added charts to normal chat + agents, improve powerpoint export, and added cards in more places

* proactive chart use, feedback table update, violations table update, improve powerpoint export

* gpt dropdown fix, violation table update

* fix tabular analysis bug, added minimize to action card in messages

* improved document analysis, conversation title creation, fix public workspace creation bug

* added tabular background process, beta file sync, source review, follow up prompt button

* deep research and updated tabular background processing

* added file sync admin ui

* recurse control, improved tab background, bug fix text paste, add drag and drop support to chat

* improved performance

* improve tab processing and file sync delete

* improve deep research reliability

* improved load more in websites

* improved website support

* file sync ui update, optional role access for file sync, and deep research, and support for visio ingest

* improved visio ingest

* fixed agent knowledge bug and depth limit

* fixed import bug

* removed auto deep research, user must select it.

* Update config.py

* added url acccess

* Update deployers

* surface search quota warning

* improved deep research ui, add web search test connection, add stop button to chat, add identities, add web search to assigned knowledge

* improved advanced search, added enable/disable chat file upload

* enabled multiple masks per message

* fix for azd windows, added databricks plugin, mcp plugin, fix tts

* workflow enable/disable, stop button, and web search defaults

* fixed min ai message width

* fix chat file upload bug, fix auto metadata extract

* add conversation pagination, add assign knowledge, add group file share approval, add time to deep research, fix workspace scope lock w/ assigned knowledge for agent

* Metadata update

* add new file sync sources, add foundry workflow

* add chart, inline image gen, improve exports

* add color edit to charts, add cosmos scale control

* background cosmos scale cadence update

* improve doc intel extraction support layout, add cosmos - manual to autoscale support

* fix scaling running in background

* auto selection (read/layout) for doc intel pdf/image extraction, cosmos scaling documentation update

* Delete deploy.zip

* Update .gitignore

* added documentation, improve chat file upload, doc intel extraction workflow, custom hero color, send mail with msgraph plugin, admins customize action configs, voice support across app,

* send email via msgraph, chat file upload to group and group multi-user conversations, group workflows, controls for send email and send calendar invite, anthropic support

* Update release_notes.md

* updated release features

* cosmos scale update, workflow per doc support, foundry protocol update, add group activity view

* updated compare ui in workflow

* support foundry api key for agent, fixed group activity new tab

* foundry agent fix, document delete fix

* added workflow agent action limit, cosmos scaling update, anthropic support for analyze and compare, foundry workflow api key fix

* group workspace download restrictions

* foundry agent and workflow update

* fix bug chat upload with agent, fix bug analyze making unrequested json

* cosmos scaling limits, agent with assigned knowledge chat file upload issue

* Delete deploy.zip

* fixed profile user_id, fix chat upload with agents, fix group workflow assignment

* fix deployer updates for cosmos, fix group manage page settings, fix route auth

* Update test_deployer_capacity_defaults.py

* assigned knowledge update

* Update .gitignore

* improvements to the agent and assigned knowledge searching and tabular use of search

* new Tableau action/plugin, new backup/restore/migrate feature

* added migration

* add agents page, added migration

* add agent page config

* updated agent catalog

* fix auto tts in chat

* fix delete, workflow, simplechat action, fix route decorator

* new security prompts

* fix csrf, fix sidebar bug

* csrf update

* fix file upload and processing

* fix file download visibility, fix deployer managed identity failure, fix public workspace bug, fix plugin payload bug, fix logging for high value info

* fix admin settings view of secrets, fix azure billing action logging, fix tts logging, fix model icon, fix docs on foundry rbac roles

* add agents page customization, fix governance toggles, fix governance for deselect all but no assigned

* add admin ability to promote agents in agents popular tab, add governance for action types

* Prepare beta branch for PR readiness

* Align CI with Python 3.12 and fix XSS guardrails
@paullizer @vivche
feat: improve tabular SK large result handling (#913)
a7147be 
* feat: improve tabular SK large result handling

Adds paginated row-returning tabular tools, safe row payload trimming, return_columns projection, a larger computed-results handoff guardrail, and Python 3.13-safe Semantic Kernel parameter annotations.

Inspired by and adapted from PRs #892 and #894 by @vivche.

Co-authored-by: vivche <214204244+vivche@users.noreply.github.com>

* docs: add tabular SK release notes

Adds v0.242.068 release notes for tabular SK large-result handling and Python 3.13 Semantic Kernel parameter compatibility.

Attributes the source ideas to PRs #892 and #894 by @vivche.

Co-authored-by: vivche <214204244+vivche@users.noreply.github.com>

---------

Co-authored-by: vivche <214204244+vivche@users.noreply.github.com>
@paullizer @vivche
Feature/tabular sk large result pagination (#914)
d6cf2ac 
* feat: improve tabular SK large result handling

Adds paginated row-returning tabular tools, safe row payload trimming, return_columns projection, a larger computed-results handoff guardrail, and Python 3.13-safe Semantic Kernel parameter annotations.

Inspired by and adapted from PRs #892 and #894 by @vivche.

Co-authored-by: vivche <214204244+vivche@users.noreply.github.com>

* docs: add tabular SK release notes

Adds v0.242.068 release notes for tabular SK large-result handling and Python 3.13 Semantic Kernel parameter compatibility.

Attributes the source ideas to PRs #892 and #894 by @vivche.

Co-authored-by: vivche <214204244+vivche@users.noreply.github.com>

* package update

---------

Co-authored-by: vivche <214204244+vivche@users.noreply.github.com>
@paullizer
Add model endpoint identity setup guide (#915)
9957673
@clarked-msft
Upgrade requests requirement (#873)
1f70347 
* Upgrade requests requirement

* Add release notes
@paullizer
Fix chat model icon avatars (#916)
7762f8f 
* Add model endpoint setup guide UI

* Fix chat model icon avatars

* Update release notes for model endpoint guidance

* Preserve agent avatar priority over model icons
@paullizer
initial
04d2c4c
@paullizer
Docs/0.250.001 (#922)
c672c04 
* add snowflake action, add single model help

* fix anthropic support for agents

* Update config.py

* updated documentation, fix masking bug, add voice assist, fix msg bug, fix stepper bug, fix web retrieval order

* Update config.py

* removing

* update gitignore
@paullizer
Record Staging as merged into Development for v0.250.001
c1c8397
@paullizer
Disable BuildKit for dev image publishers (#924)
7544947 
* Disable BuildKit for dev image publishers

* initial
@paullizer @clarked-msft
@Bionic711 @vivche @eldong @Patrick-Davis-MSFT @nadoylemsft @cjackson202 @SteveCInVA @Xeelee33 @eldongormsen @chadpalmer
v0.250.001 (#923)
c047d34 
* Removed unneeded folder and document

* Revert terraform main.tf to upstream/Development version

* updated the logging logic when running retention delete with archiving enabled (#642)

* Corrected version to 0.236.011 (#645)

* v0.237.001 (#649)

* Use Microsoft python base image

* Add python ENV vars

* Add python ENV vars

* Install deps to systme

* Add temp dir to image and pip conf support

* Add custom-ca-certificates dir

* Logo bug fix (#654)

* release note updating for github coplilot

* fixed logo bug issue

* added 2,3,4,5,6,14 days to rentention policy

* added retention policy time updates

* Rentention policy (#657)

* Critical Retention Policy Deletion Fix

* Create RETENTION_POLICY_NULL_LAST_ACTIVITY_FIX.md

* Added ServiceNow support for create and publish article.  Including readme for configuration for 2 agents.

* Replace actual servicenow instance name with generic name in the readme file

* Changed version number in ServiceNow readme files to 0.237.005 since this pull request has latest changes from v0.237.004 Development branch

* Enhance ServiceNow agent for managing new KB article creation

* Added readme and open ai specs and agent instructions to support ServiceNow asset management

* Remove any references to actual ServiceNow instances

* fixed retention policy runtime bug and sidebar bug (#672)

* Fix: Windows Unicode encoding issue for video uploads (#662)

- Added explicit UTF-8 encoding when reading file content on Windows
- Prevents UnicodeDecodeError when processing non-ASCII filenames
- Ensures consistent file handling across different operating systems

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md (#675)

Co-authored-by: vivche <vivche@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add custom subdomain support for OpenAI and Speech Service in Terraform (#558)

* Add custom subdomain support for OpenAI and Speech Service in Terraform

- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* 0.237.006 (#676)

* Update chat-sidebar-conversations.js

* 0.237.006

* docs: Update release notes for ServiceNow integration and bug fixes

- Add comprehensive ServiceNow integration documentation entry
- Document 4 critical bug fixes:
  - OpenAPI Basic Authentication credential parsing fix
  - Group Action OAuth schema merging fix
  - Group Agent loading in per-user kernel mode fix
  - Azure AI Search test connection fix
- Move debug_print import to top-level in openapi_plugin_factory.py

Version: 0.237.005

* Update release_notes.md

* resolve conflict

* fixed sidebar race condition (#679)

* fix the version number in config.py

* Security: Restrict group agent loading to active group only

WHAT:
- Changed group agent loading from ALL user groups to ACTIVE group only
- Added security validation to prevent cross-group agent access
- Updated import from get_user_groups to require_active_group

WHY:
- Prevents users from accessing group agents outside their active group
- Fixes a security gap where selecting a group agent bypassed the active-group check
- Aligns behavior with the intended single-active-group security model

FILES:
- semantic_kernel_loader.py (approx. lines 11801280)
- GROUP_AGENT_LOADING_FIX.md

VERSION:
- 0.237.008

* Fixed an instruction error that caused semantic kernel to fall back to default agent

* Fixed! The issue was caused by duplicated code blocks (#683)

* Manage group frontend bug (#684)

* Fixed! The issue was caused by duplicated code blocks

* updated release notes

* initial feature add

* added tag endpoint

* return unused tags too

* Bicepfix (#690)

* v0.237.006 (#678)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from query

* rmv missed merge lines

* fix for AL

* upd for consistency testing

* upd abp to community

* fix copilot findings #1

* fix plotting conflict

* fix exception handling

* fix static max function invokes

* rmv unneeded decl

* rmv unneeded imports

* fix grouping dimensions

* fix abp copilot suggestions #2

* simplify methods for message reload

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

* upd pipeline to include branch name in image

* add abp to deploy

* upd instructions name/rmv abp from deploy

* fix pipeline

* mov back to Comm Cust for main inclusion

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Security/container build (#549)

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/speech managed identity (#543)

* Bugfix - deleted duplicate enable_external_healthcheck entry

* Feature - updated Speech Service to use Managed Identity in addition to the key, added MAG functionality via Azure Speech SDK since the Fast Transcription API is not available in MAG, updated Admin Setup Walkthrough so it goes to the right place in the settings when Next is clicked, updated Speech requirements in Walkthrough, rewrote Admin Configuration docs, updated/corrected Managed Identity roles in Setup Instructions Special docs.

* Update application/single_app/templates/admin_settings.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_settings.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Paul Lizer <paullizer@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

* Control center (#567)

* added group status (active, locked, upload disabled, and inactive)

Adds Azure Billing Plugin in Community Customizations

* added bulk member upload via csv for groups

* add document metadata modified activity log tracking

* activity logging for members deleted from groups

* added group activity timeline

* added notification system

* added notifications for document upload to workspaces

* fixed badge sizing

* fixed url link

* fixed badge to not show with zero notifications

* Updated notification system

* Updated approval system

* updated approval workflow

* updated notification workflow

* Fixed set active bug on my public workspace page

* Added user retention policy, updated user profile page with dashboards, retention config, and more.

* adding speed to text for chat UI

* updated the speech wave form and input field

* updated to transcribe entire recording

* fixed bug creating new conversation with auto-send

* add mic permissions

* added stream token tracking

* Added public workspace reporting

* Updated AI search sizing analysis

* added management for public workspaces

* improved public workspace management includes stats and bulk actions

* updated groups dashboard for owners and admins with stats and bulk actions

* added voice for ai to talk with users in chats

* Auto Voice Response

* for speech service, added 429 randomized response pattern to prevent thunder herding

* updated admin settings for speech services and fixed dark mode for raw log viewing

* updated video extraction card

* Added Control Center Admin and Dashboard Reader roles

* updated feedback and safety decorators so admins work unless required then those roles must be used

* Updated and Validated logic for admin roles; control center, safety, and feedback

* added support for control center admin and dashboard reader

* Development (#566)

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* updated tool tip to better inform user on status of ai response

* improve query parameters detection for swagger

* updated visual cue showing the ai is talking to the user

* moved duplicates to shared js

* replaced alert with toast.

* fixed and added log_event to exceptions

* added @user_required and improved swagger generation

* Update route_frontend_profile.py

* fixed swagger generation bug on affecting two apis

* returned keyvault to admin settings ui

* Fixed bug when running local js

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* Adding release notes

* fixed debug_debug_print

* Updated README

* Update README.md

* accepted changes

* removed files

* GitHub Actions workflow that runs Python compilation checks on all Python files in the single_app

* Upated to v0.235.002

* removed debug test file

* Updated to v0.235.003

* Update python-syntax-check.yml

* fixed disable group creation bug

* fixed bug not showing raw activity log for groups

* fixed control center access to not require admin role when enabling controlcenteradmin role

* fix documentation

* Update release_notes.md

* Update README.md

* added click restrictions to top items in control center

* bug fix - fixed group member select after search, group member removal, group member role update, and approve/reject request

* Retention execution activity log (#600)

* retention execution logging bug fix

* debug timer reset with admin save bug fix

* Create test_debug_logging_timer_preservation.py

* fixed file processing logic prevent runtime execution

* fixed bug processing execution against personal documents

* removed test logging

* Retention Policy Document Deletion Fix

* Improve execution logic for rentention policy

Now uses retention_policy_next_run timestamp - Compares current time against the stored next scheduled time. If current time >= next scheduled time, it runs.

Reduced check interval from 1 hour to 5 minutes - More responsive scheduling, ensures it catches the scheduled time promptly.

Better fallback logic - If next_run can't be parsed, falls back to checking last_run (23-hour threshold).

Runs immediately if never run before - If there's no last_run or next_run, it will execute on the first check.

* Updated branch flow

* added log_event to exceptions

* enforce-branch-flow

* v0.235.025 (#609)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding …
github-code-quality[bot]
github-code-quality Bot found potential problems Jun 25, 2026
View reviewed changes
Comment thread application/single_app/static/js/chat/chat-citations.js Dismissed
Comment thread application/single_app/static/js/profile/profile-tabs.js Dismissed
Comment thread application/single_app/static/js/profile/profile-tabs.js Dismissed
paullizer and others added 2 commits June 24, 2026 22:18
@paullizer @Patrick-Davis-MSFT
@nadoylemsft @cjackson202 @Bionic711 @SteveCInVA @Xeelee33 @clarked-msft @vivche @eldong @eldongormsen @chadpalmer
Record main as merged into Development for v0.250.001 (#926)
44d35b5 
* branch workflow update (#864)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from query

* rmv missed merge lines

* fix for AL

* upd for consistency testing

* upd abp to community

* fix copilot findings #1

* fix plotting conflict

* fix exception handling

* fix static max function invokes

* rmv unneeded decl

* rmv unneeded imports

* fix grouping dimensions

* fix abp copilot suggestions #2

* simplify methods for message reload

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

* upd pipeline to include branch name in image

* add abp to deploy

* upd instructions name/rmv abp from deploy

* fix pipeline

* mov back to Comm Cust for main inclusion

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Security/container build (#549)

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/speech managed identity (#543)

* Bugfix - deleted duplicate enable_external_healthcheck entry

* Feature - updated Speech Service to use Managed Identity in addition to the key, added MAG functionality via Azure Speech SDK since the Fast Transcription API is not available in MAG, updated Admin Setup Walkthrough so it goes to the right place in the settings when Next is clicked, updated Speech requirements in Walkthrough, rewrote Admin Configuration docs, updated/corrected Managed Identity roles in Setup Instructions Special docs.

* Update application/single_app/templates/admin_settings.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_settings.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Paul Lizer <paullizer@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

* Control center (#567)

* added group status (active, locked, upload disabled, and inactive)

Adds Azure Billing Plugin in Community Customizations

* added bulk member upload via csv for groups

* add document metadata modified activity log tracking

* activity logging for members deleted from groups

* added group activity timeline

* added notification system

* added notifications for document upload to workspaces

* fixed badge sizing

* fixed url link

* fixed badge to not show with zero notifications

* Updated notification system

* Updated approval system

* updated approval workflow

* updated notification workflow

* Fixed set active bug on my public workspace page

* Added user retention policy, updated user profile page with dashboards, retention config, and more.

* adding speed to text for chat UI

* updated the speech wave form and input field

* updated to transcribe entire recording

* fixed bug creating new conversation with auto-send

* add mic permissions

* added stream token tracking

* Added public workspace reporting

* Updated AI search sizing analysis

* added management for public workspaces

* improved public workspace management includes stats and bulk actions

* updated groups dashboard for owners and admins with stats and bulk actions

* added voice for ai to talk with users in chats

* Auto Voice Response

* for speech service, added 429 randomized response pattern to prevent thunder herding

* updated admin settings for speech services and fixed dark mode for raw log viewing

* updated video extraction card

* Added Control Center Admin and Dashboard Reader roles

* updated feedback and safety decorators so admins work unless required then those roles must be used

* Updated and Validated logic for admin roles; control center, safety, and feedback

* added support for control center admin and dashboard reader

* Development (#566)

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* updated tool tip to better inform user on status of ai response

* improve query parameters detection for swagger

* updated visual cue showing the ai is talking to the user

* moved duplicates to shared js

* replaced alert with toast.

* fixed and added log_event to exceptions

* added @user_required and improved swagger generation

* Update route_frontend_profile.py

* fixed swagger generation bug on affecting two apis

* returned keyvault to admin settings ui

* Fixed bug when running local js

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* Adding release notes

* fixed debug_debug_print

* Updated README

* Update README.md

* accepted changes

* removed files

* GitHub Actions workflow that runs Python compilation checks on all Python files in the single_app

* Upated to v0.235.002

* removed debug test file

* Updated to v0.235.003

* Update python-syntax-check.yml

* fixed disable group creation bug

* fixed bug not showing raw activity log for groups

* fixed control center access to not require admin role when enabling controlcenteradmin role

* fix documentation

* Update release_notes.md

* Update README.md

* added click restrictions to top items in control center

* bug fix - fixed group member select after search, group member removal, group member role update, and approve/reject request

* Retention execution activity log (#600)

* retention execution logging bug fix

* debug timer reset with admin save bug fix

* Create test_debug_logging_timer_preservation.py

* fixed file processing logic prevent runtime execution

* fixed bug processing execution against personal documents

* removed test logging

* Retention Policy Document Deletion Fix

* Improve execution logic for rentention policy

Now uses retention_policy_next_run timestamp - Compares current time against the stored next scheduled time. If current time >= next scheduled time, it runs.

Reduced check interval from 1 hour to 5 minutes - More responsive scheduling, ensures it catches the scheduled time promptly.

Better fallback logic - If next_run can't be parsed, falls back to checking last_run (23-hour threshold).

Runs immediately if never run before - If there's no last_run or next_run, it will execute on the first check.

* Updated branch flow

* added log_event to exceptions

* enforce-branch-flow

* v0.235.025 (#609)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate g…
@paullizer @clarked-msft
@Bionic711 @vivche @eldong @Patrick-Davis-MSFT @nadoylemsft @cjackson202 @SteveCInVA @Xeelee33 @eldongormsen @chadpalmer
Sync Development to Staging after main history merge (#927)
0082284 
* Revert terraform main.tf to upstream/Development version

* updated the logging logic when running retention delete with archiving enabled (#642)

* Corrected version to 0.236.011 (#645)

* v0.237.001 (#649)

* Use Microsoft python base image

* Add python ENV vars

* Add python ENV vars

* Install deps to systme

* Add temp dir to image and pip conf support

* Add custom-ca-certificates dir

* Logo bug fix (#654)

* release note updating for github coplilot

* fixed logo bug issue

* added 2,3,4,5,6,14 days to rentention policy

* added retention policy time updates

* Rentention policy (#657)

* Critical Retention Policy Deletion Fix

* Create RETENTION_POLICY_NULL_LAST_ACTIVITY_FIX.md

* Added ServiceNow support for create and publish article.  Including readme for configuration for 2 agents.

* Replace actual servicenow instance name with generic name in the readme file

* Changed version number in ServiceNow readme files to 0.237.005 since this pull request has latest changes from v0.237.004 Development branch

* Enhance ServiceNow agent for managing new KB article creation

* Added readme and open ai specs and agent instructions to support ServiceNow asset management

* Remove any references to actual ServiceNow instances

* fixed retention policy runtime bug and sidebar bug (#672)

* Fix: Windows Unicode encoding issue for video uploads (#662)

- Added explicit UTF-8 encoding when reading file content on Windows
- Prevents UnicodeDecodeError when processing non-ASCII filenames
- Ensures consistent file handling across different operating systems

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md (#675)

Co-authored-by: vivche <vivche@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add custom subdomain support for OpenAI and Speech Service in Terraform (#558)

* Add custom subdomain support for OpenAI and Speech Service in Terraform

- Added custom_subdomain_name to OpenAI resource for managed identity authentication
- Created Speech Service resource with custom subdomain configuration
- Added RBAC role assignments for Speech Service (Managed Identity and App Service MI)
- Includes Cognitive Services Speech User and Speech Contributor roles
- Documentation: Azure Speech managed identity setup guide

* Update docs/how-to/azure_speech_managed_identity_manul_setup.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Chen, Vivien <Vivien.Chen+ecolab@ecolab.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* 0.237.006 (#676)

* Update chat-sidebar-conversations.js

* 0.237.006

* docs: Update release notes for ServiceNow integration and bug fixes

- Add comprehensive ServiceNow integration documentation entry
- Document 4 critical bug fixes:
  - OpenAPI Basic Authentication credential parsing fix
  - Group Action OAuth schema merging fix
  - Group Agent loading in per-user kernel mode fix
  - Azure AI Search test connection fix
- Move debug_print import to top-level in openapi_plugin_factory.py

Version: 0.237.005

* Update release_notes.md

* resolve conflict

* fixed sidebar race condition (#679)

* fix the version number in config.py

* Security: Restrict group agent loading to active group only

WHAT:
- Changed group agent loading from ALL user groups to ACTIVE group only
- Added security validation to prevent cross-group agent access
- Updated import from get_user_groups to require_active_group

WHY:
- Prevents users from accessing group agents outside their active group
- Fixes a security gap where selecting a group agent bypassed the active-group check
- Aligns behavior with the intended single-active-group security model

FILES:
- semantic_kernel_loader.py (approx. lines 11801280)
- GROUP_AGENT_LOADING_FIX.md

VERSION:
- 0.237.008

* Fixed an instruction error that caused semantic kernel to fall back to default agent

* Fixed! The issue was caused by duplicated code blocks (#683)

* Manage group frontend bug (#684)

* Fixed! The issue was caused by duplicated code blocks

* updated release notes

* initial feature add

* added tag endpoint

* return unused tags too

* Bicepfix (#690)

* v0.237.006 (#678)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from query

* rmv missed merge lines

* fix for AL

* upd for consistency testing

* upd abp to community

* fix copilot findings #1

* fix plotting conflict

* fix exception handling

* fix static max function invokes

* rmv unneeded decl

* rmv unneeded imports

* fix grouping dimensions

* fix abp copilot suggestions #2

* simplify methods for message reload

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

* upd pipeline to include branch name in image

* add abp to deploy

* upd instructions name/rmv abp from deploy

* fix pipeline

* mov back to Comm Cust for main inclusion

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Security/container build (#549)

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/speech managed identity (#543)

* Bugfix - deleted duplicate enable_external_healthcheck entry

* Feature - updated Speech Service to use Managed Identity in addition to the key, added MAG functionality via Azure Speech SDK since the Fast Transcription API is not available in MAG, updated Admin Setup Walkthrough so it goes to the right place in the settings when Next is clicked, updated Speech requirements in Walkthrough, rewrote Admin Configuration docs, updated/corrected Managed Identity roles in Setup Instructions Special docs.

* Update application/single_app/templates/admin_settings.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_settings.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Paul Lizer <paullizer@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

* Control center (#567)

* added group status (active, locked, upload disabled, and inactive)

Adds Azure Billing Plugin in Community Customizations

* added bulk member upload via csv for groups

* add document metadata modified activity log tracking

* activity logging for members deleted from groups

* added group activity timeline

* added notification system

* added notifications for document upload to workspaces

* fixed badge sizing

* fixed url link

* fixed badge to not show with zero notifications

* Updated notification system

* Updated approval system

* updated approval workflow

* updated notification workflow

* Fixed set active bug on my public workspace page

* Added user retention policy, updated user profile page with dashboards, retention config, and more.

* adding speed to text for chat UI

* updated the speech wave form and input field

* updated to transcribe entire recording

* fixed bug creating new conversation with auto-send

* add mic permissions

* added stream token tracking

* Added public workspace reporting

* Updated AI search sizing analysis

* added management for public workspaces

* improved public workspace management includes stats and bulk actions

* updated groups dashboard for owners and admins with stats and bulk actions

* added voice for ai to talk with users in chats

* Auto Voice Response

* for speech service, added 429 randomized response pattern to prevent thunder herding

* updated admin settings for speech services and fixed dark mode for raw log viewing

* updated video extraction card

* Added Control Center Admin and Dashboard Reader roles

* updated feedback and safety decorators so admins work unless required then those roles must be used

* Updated and Validated logic for admin roles; control center, safety, and feedback

* added support for control center admin and dashboard reader

* Development (#566)

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* updated tool tip to better inform user on status of ai response

* improve query parameters detection for swagger

* updated visual cue showing the ai is talking to the user

* moved duplicates to shared js

* replaced alert with toast.

* fixed and added log_event to exceptions

* added @user_required and improved swagger generation

* Update route_frontend_profile.py

* fixed swagger generation bug on affecting two apis

* returned keyvault to admin settings ui

* Fixed bug when running local js

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* Adding release notes

* fixed debug_debug_print

* Updated README

* Update README.md

* accepted changes

* removed files

* GitHub Actions workflow that runs Python compilation checks on all Python files in the single_app

* Upated to v0.235.002

* removed debug test file

* Updated to v0.235.003

* Update python-syntax-check.yml

* fixed disable group creation bug

* fixed bug not showing raw activity log for groups

* fixed control center access to not require admin role when enabling controlcenteradmin role

* fix documentation

* Update release_notes.md

* Update README.md

* added click restrictions to top items in control center

* bug fix - fixed group member select after search, group member removal, group member role update, and approve/reject request

* Retention execution activity log (#600)

* retention execution logging bug fix

* debug timer reset with admin save bug fix

* Create test_debug_logging_timer_preservation.py

* fixed file processing logic prevent runtime execution

* fixed bug processing execution against personal documents

* removed test logging

* Retention Policy Document Deletion Fix

* Improve execution logic for rentention policy

Now uses retention_policy_next_run timestamp - Compares current time against the stored next scheduled time. If current time >= next scheduled time, it runs.

Reduced check interval from 1 hour to 5 minutes - More responsive scheduling, ensures it catches the scheduled time promptly.

Better fallback logic - If next_run can't be parsed, falls back to checking last_run (23-hour threshold).

Runs immediately if never run before - If there's no last_run or next_run, it will execute on the first check.

* Updated branch flow

* added log_event to exceptions

* enforce-branch-flow

* v0.235.025 (#609)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embeddin…
paullizer and others added 5 commits June 24, 2026 22:27
@paullizer @Patrick-Davis-MSFT
@nadoylemsft @cjackson202 @Bionic711 @SteveCInVA @Xeelee33 @clarked-msft @vivche @eldong @eldongormsen @chadpalmer
Repair v0.250.001 release branch ancestry (#928)
a6f118a 
* branch workflow update (#864)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from query

* rmv missed merge lines

* fix for AL

* upd for consistency testing

* upd abp to community

* fix copilot findings #1

* fix plotting conflict

* fix exception handling

* fix static max function invokes

* rmv unneeded decl

* rmv unneeded imports

* fix grouping dimensions

* fix abp copilot suggestions #2

* simplify methods for message reload

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

* upd pipeline to include branch name in image

* add abp to deploy

* upd instructions name/rmv abp from deploy

* fix pipeline

* mov back to Comm Cust for main inclusion

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Security/container build (#549)

* upd dockerfile to google distroless

* add pipelines

* add modifications to container

* upd to build

* add missing arg

* add arg for major/minor/patch python version

* upd python paths and pip install

* add perms to /app for user

* chg back to root

* rmv python3

* rmv not built python

* add shared

* add path and home

* upd for stdlib paths

* fix user input filesystem path vulns

* fix to consecutive dots

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/speech managed identity (#543)

* Bugfix - deleted duplicate enable_external_healthcheck entry

* Feature - updated Speech Service to use Managed Identity in addition to the key, added MAG functionality via Azure Speech SDK since the Fast Transcription API is not available in MAG, updated Admin Setup Walkthrough so it goes to the right place in the settings when Next is clicked, updated Speech requirements in Walkthrough, rewrote Admin Configuration docs, updated/corrected Managed Identity roles in Setup Instructions Special docs.

* Update application/single_app/templates/admin_settings.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_settings.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update application/single_app/functions_documents.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Paul Lizer <paullizer@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

* Control center (#567)

* added group status (active, locked, upload disabled, and inactive)

Adds Azure Billing Plugin in Community Customizations

* added bulk member upload via csv for groups

* add document metadata modified activity log tracking

* activity logging for members deleted from groups

* added group activity timeline

* added notification system

* added notifications for document upload to workspaces

* fixed badge sizing

* fixed url link

* fixed badge to not show with zero notifications

* Updated notification system

* Updated approval system

* updated approval workflow

* updated notification workflow

* Fixed set active bug on my public workspace page

* Added user retention policy, updated user profile page with dashboards, retention config, and more.

* adding speed to text for chat UI

* updated the speech wave form and input field

* updated to transcribe entire recording

* fixed bug creating new conversation with auto-send

* add mic permissions

* added stream token tracking

* Added public workspace reporting

* Updated AI search sizing analysis

* added management for public workspaces

* improved public workspace management includes stats and bulk actions

* updated groups dashboard for owners and admins with stats and bulk actions

* added voice for ai to talk with users in chats

* Auto Voice Response

* for speech service, added 429 randomized response pattern to prevent thunder herding

* updated admin settings for speech services and fixed dark mode for raw log viewing

* updated video extraction card

* Added Control Center Admin and Dashboard Reader roles

* updated feedback and safety decorators so admins work unless required then those roles must be used

* Updated and Validated logic for admin roles; control center, safety, and feedback

* added support for control center admin and dashboard reader

* Development (#566)

* Banner text color picker from Vivien (#555)

* Classification text color picker

* Line endings

* Remove opencensus

* Add flask instrumentation

* Add troubleshooting doc

* Add troubleshooting doc

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* updated tool tip to better inform user on status of ai response

* improve query parameters detection for swagger

* updated visual cue showing the ai is talking to the user

* moved duplicates to shared js

* replaced alert with toast.

* fixed and added log_event to exceptions

* added @user_required and improved swagger generation

* Update route_frontend_profile.py

* fixed swagger generation bug on affecting two apis

* returned keyvault to admin settings ui

* Fixed bug when running local js

---------

Co-authored-by: Ed Clark <107473135+clarked-msft@users.noreply.github.com>
Co-authored-by: Ed Clark <clarked@microsoft.com>
Co-authored-by: Bionic711 <13358952+Bionic711@users.noreply.github.com>

* Adding release notes

* fixed debug_debug_print

* Updated README

* Update README.md

* accepted changes

* removed files

* GitHub Actions workflow that runs Python compilation checks on all Python files in the single_app

* Upated to v0.235.002

* removed debug test file

* Updated to v0.235.003

* Update python-syntax-check.yml

* fixed disable group creation bug

* fixed bug not showing raw activity log for groups

* fixed control center access to not require admin role when enabling controlcenteradmin role

* fix documentation

* Update release_notes.md

* Update README.md

* added click restrictions to top items in control center

* bug fix - fixed group member select after search, group member removal, group member role update, and approve/reject request

* Retention execution activity log (#600)

* retention execution logging bug fix

* debug timer reset with admin save bug fix

* Create test_debug_logging_timer_preservation.py

* fixed file processing logic prevent runtime execution

* fixed bug processing execution against personal documents

* removed test logging

* Retention Policy Document Deletion Fix

* Improve execution logic for rentention policy

Now uses retention_policy_next_run timestamp - Compares current time against the stored next scheduled time. If current time >= next scheduled time, it runs.

Reduced check interval from 1 hour to 5 minutes - More responsive scheduling, ensures it catches the scheduled time promptly.

Better fallback logic - If next_run can't be parsed, falls back to checking last_run (23-hour threshold).

Runs immediately if never run before - If there's no last_run or next_run, it will execute on the first check.

* Updated branch flow

* added log_event to exceptions

* enforce-branch-flow

* v0.235.025 (#609)

* creating workflows

* fraud analyssis

* support agents

* update

* fix

* updated demo

* Swagger lite (#469)

* Development (#467)

* upgrade to v0.229.060 (#459)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* docs demos for public workspaces

* V0.229 bug fixes (v0.229.019) (#448)

* Development (#445)

* Update release notes to show support for GPT-5

* Documented support for gpt-image-1

* Update config.py

* remove documentation folder

* Documentation and message table support (#444)

* Develop demo docs and import markdown table support

* fixed enhanced citations for groups and public workspaces

* Updated to support showing public workspaces in scope

* Update config.py

* fix docs

* Updated RELEASE_NOTES

* video indexer config details, doc intel test button fix, move multimedia configs to search and extract

* improved header security

* updated versions

* moved

* Update EXTERNAL_HEALTH_CHECK_DUPLICATION_FIX.md

* added pdfs

* v0.229.019 bug fixes upgrade to v0.229.058 (#452)

* all urls in chat open in new tabs

* consolidated admin settings for improved navigation

* added left hand nav admin settings menus

* added left hand menu options for workspaces

* Added debug logging to video indexer processes

* readme and functional test

* Workspace Scope in Chat affects Prompts

* Create WORKSPACE_SCOPE_PROMPTS_FIX.md

* time based turn off for debug and file process logging

* improve saving in admin settings

* update to v0.229.058

* Update RELEASE_NOTES.md

* Update RELEASE_NOTES.md

* Popup modal for Health Check config

* Added Health Check config guide

* Chat page top nav bug (#458)

* initial fix

* fixed top nav chat up bug

* notes for v0.229.060

* file location fix

* Update config.py

* Update RELEASE_NOTES.md

* moved to correct location

* Fixed enhanced citations CSP bug

Simple Chat implemented improved security which negatively impacted enhanced citations.

* Updated release notes

* updated version and tests

* swagger support for all endpoints and added swagger search

* added wide screen support for chats when collapsing side bar

* v0.230.001 features

* adding support for xlsm, Macro Excel files.

* moved into features

* initial

* added readme

* removed html code

* Update config.py (#477)

Updated else if for AUTHORITY

* Initial Setup for Pages documentation (#479)

* setup folders and base files

* setting up files

* architecture diagrams

* updated to libdoc

* libdoc updates

* updating side bar

* removed loops

* editing side bar

* Created Simple Chat Jekyll theme

* Update config.py (#477) (#478)

Updated else if for AUTHORITY

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* Updating architectures

* Update README.md

---------

Co-authored-by: Patrick C Davis <82388365+Patrick-Davis-MSFT@users.noreply.github.com>

* initial

* added to base

* adding real data endpoints

* Update route_backend_control_center.py

* added individual charts

* fix for bug 485

* added document metrics

* added links to control center

* debug

* added date

* fixed bugs due to branch descrepancies

* added Azure SQL Driver Docker File

* added documentation for docker_fileSession updates

* Redis Managed Identity Azure Government Support Changes

* Stop tracking ignored folders

* updated gitignore

* added sort by to table for user management

* storage account size processing

* Front end now shows storage account sizing

* export user management list to csv

* adding group management

* fixing swagger generation

* fix

* Added inline dynamic property generation

* added YAML support

* Improved muiltform vs app/json detection

* added Control Center Admin role

ControlCenterAdmin

* ai search sizing is working for groups

* group refresh fixed

* added group data fix

* group table refresh

* updated export to include group docs

* adding public workspace management

* removed sample data and consolidated row generators

* Changed both caching helper functions to use the existing update_document() function from functions_documents.py instead of direct upsert.

* removed workflow, will work on that in different branch

* Document Set Fingerprinting, Scope-Aware Cache Key Generation, Event-Based Invalidation

I've successfully implemented Document Set Fingerprint + Event-Based Cache Invalidation with deterministic sorting and Score Normalization.

* added debug logging

* setup cache feature and ttl time to admin app settings

* removed cosmos level ttl

* Keyvault for secrets (#492)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/remove abp for pr (#510)

* add crude keyvault base impl

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* add ui trigger word and get agent helper

* adds safe calls for akv functions

* add keyvault to global actions loading

* rmv secret leak via logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add security tab UI + key vault UI

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Feature/group agents actions (#521)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add group agents/actions

* add branch for testing/rmv old branch

* bug fixes, group agent modifications, rmv client validation

* rmv ajv

* upd from copilot

---------

Co-authored-by: Bionic711 <nadoyle@microsoft.com>

* Add cosmos activity logs container configuration

* incorporate branch updates Add 372 fix 489

* Support deployment via AZD UP (#530)

* Update devcontainer configuration for support of AZD

* Move to module based bicep files

* Add Azure deployment configuration and update Bicep modules for service outputs

* Enhance Azure deployment process by adding predeploy hooks for Docker image management and updating Bicep modules to include managed identity client ID and container registry outputs.

* Add deployment script for creating and storing Azure AD client secret in Key Vault

* Update Azure Dev CLI feature version to latest in devcontainer configuration

* Remove deprecated Bicep files and parameter configurations for cleaner deployment structure

* Refactor Bicep modules for improved diagnostics and role assignments

- Updated appService.bicep to conditionally import diagnostic settings based on enableDiagLogging parameter.
- Changed Azure Cosmos DB authentication type to managed identity and removed key-based authentication settings.
- Enhanced appServiceAuthentication.bicep by removing unnecessary parameters and configuring Key Vault reference for client secret.
- Modified appServicePlan.bicep to conditionally import diagnostic settings.
- Refactored azureContainerRegistry-existing.bicep to deploy role assignment to the ACR's resource group.
- Updated azureContainerRegistry.bicep to conditionally import diagnostic settings.
- Enhanced contentSafety.bicep with conditional diagnostic settings import.
- Updated cosmosDb.bicep to include a new database and container, and added role assignments for managed identity.
- Refactored documentIntelligence.bicep to conditionally import diagnostic settings.
- Enhanced enterpriseApplication.bicep by adding additional required resource access scopes.
- Updated keyVault.bicep to conditionally import diagnostic settings and adjusted enterprise app parameters.
- Refactored openAI.bicep to conditionally import diagnostic settings.
- Enhanced redisCache.bicep with conditional diagnostic settings import.
- Updated search.bicep to conditionally import diagnostic settings.
- Refactored speechService.bicep to conditionally import diagnostic settings.
- Enhanced storageAccount.bicep with conditional diagnostic settings import.
- Added main.parameters.json for parameter management.
- Introduced azureContainerRegistry-roleAssignment.bicep for managing ACR role assignments.

* Add custom subdomain names for document intelligence, OpenAI, and speech services

* Fix casing for hostingMode property in search service configuration

* Enhance storage account configuration by enabling hierarchical namespace and setting public access to 'None' for document containers

* Add enterprise app permissions module for resource access management

* Fixed ExternalApi configuration to valid guid and set value to a unique name

* Add Init Script to Configure Entra Application

* Fix spelling error

* fix failure in hostingMode value

* configure managed identity for contentSafety

* update readme to support new AZD deployment solution

* Video Indexer, Multi-Modal Enhancements, Scope Bug

## PR Summary: Video Indexer Multi-Modal Enhancements

### Overview
This PR introduces significant enhancements to video processing and image analysis capabilities, focusing on multi-modal AI features and improved metadata handling. **Version updated from 0.233.167 to 0.233.172**.

### 🎯 Key Features

#### 1. **Multi-Modal Vision Analysis for Images**
- Added AI-powered vision analysis for uploaded images using GPT-4 Vision or similar models
- Extracts comprehensive image insights including:
  - AI-generated descriptions
  - Object detection
  - Text extraction from images (OCR)
  - Detailed visual analysis
- New admin setting: `enable_multimodal_vision` to control feature availability
- Vision analysis results stored in document metadata and included in AI Search indexing
- Connection testing endpoint added for vision model validation

#### 2. **Enhanced Document Metadata Citations**
- Implemented metadata-based citations that surface document keywords, abstracts, and vision analysis
- New citation types displayed with distinct visual indicators:
  - **Keywords**: Tagged with `bi-tags` icon, labeled as "Metadata"
  - **Abstract**: Document summaries included as contextual citations
  - **Vision Analysis**: AI-generated image insights labeled as "AI Vision"
- Metadata content passed to AI models as additional context for more informed responses
- Special modal view for metadata citations (separate from standard document citations)

#### 3. **Image Message UI Improvements**
- Enhanced display for user-uploaded images vs AI-generated images
- Added "View Text" button for uploaded images with extracted content or vision analysis
- Collapsible info sections showing:
  - Extracted OCR text from Document Intelligence
  - AI Vision Analysis results
- Proper avatar distinction between uploaded and generated images
- Improved metadata tracking with `is_user_upload` flag

#### 4. **Video Indexer Configuration Updates**
- **BREAKING CHANGE**: Removed API key authentication support
- Now exclusively uses **Managed Identity authentication** for Video Indexer
- Updated admin UI documentation to guide managed identity setup:
  - Enable system-assigned managed identity on App Service
  - Assign "Video Indexer Restricted Viewer" role
  - Configure required ARM settings (subscription ID, resource group, account name)
- Improved validation for required Video Indexer settings
- Enhanced error messaging for missing configuration

#### 5. **Search Scope Improvements**
- Fixed search behavior when `document_scope='all'` to properly include group documents
- Added `active_group_id` to search context when document scope is 'all' and groups are enabled
- Conditional group index searching - only queries group index when `active_group_id` is present
- Prevents unnecessary searches and potential errors when groups aren't in use

#### 6. **Image Context in Conversation History**
- Enhanced conversation history to include rich image context for AI models
- Extracts and includes:
  - OCR text from Document Intelligence (up to max content length)
  - AI Vision analysis (description, objects, text)
  - Structured prompt formatting for multimodal understanding
- **Important**: Base64 image data excluded from conversation history to prevent token overflow
- Only metadata and extracted insights passed to models for efficient token usage

### 🔧 Technical Improvements

#### Backend Changes
- **route_backend_chats.py**: 
  - Added metadata citation extraction logic (~150 lines)
  - Enhanced conversation history building for image uploads
  - Improved search argument handling for group contexts
  
- **functions_documents.py**:
  - New `analyze_image_with_vision_model()` function for AI vision analysis
  - Enhanced `get_document_metadata_for_citations()` integration
  - Vision analysis now runs BEFORE chunk saving to include insights in AI Search indexing
  - Removed redundant blob storage for vision JSON (stored in document metadata)
  
- **route_backend_settings.py**:
  - New `_test_multimodal_vision_connection()` endpoint for testing vision models
  - Supports both APIM and direct Azure OpenAI endpoints
  - Test uses 1x1 pixel sample image for validation

- **functions_search.py**:
  - Added conditional logic for group search execution
  - Prevents empty `active_group_id` from causing search errors

#### Frontend Changes
- **chat-messages.js** (~275 lines changed):
  - Enhanced `appendMessage()` to handle uploaded image metadata
  - New `toggleImageInfo()` functionality for expandable image details
  - Improved citation rendering with metadata type indicators
  - Debug logging for image message processing
  
- **chat-citations.js** (~70 lines added):
  - New `showMetadataModal()` function for displaying keywords/abstracts/vision analysis
  - Enhanced citation click handling to detect metadata citations
  - Separate modal styling and behavior for metadata vs document citations

- **admin_settings.html**:
  - Complete redesign of Video Indexer configuration section
  - Removed all API key references
  - Added managed identity setup instructions with step-by-step guidance
  - Updated configuration display to show resource group and subscription ID

- **_video_indexer_info.html**:
  - Updated modal content to clarify managed identity requirement
  - Added warning banner about authentication type
  - Enhanced configuration display with ARM resource details

### 📊 Files Changed
- **16 files** modified
- **+1,063 insertions**, **-412 deletions**
- Net change: **+651 lines**

### 🧪 Testing Considerations
- Test multi-modal vision analysis with various image types
- Validate metadata citations appear correctly in chat responses
- Verify Video Indexer works with managed identity authentication
- Test search scope behavior with and without groups enabled
- Validate image upload UI shows extracted text and vision analysis
- Confirm conversation history properly handles image context without token overflow

### 🔐 Security & Performance
- Managed identity authentication improves security posture (no stored API keys)
- Image base64 data excluded from conversation history prevents token exhaustion
- Metadata citations add minimal overhead while providing rich context
- Vision analysis runs efficiently during document processing pipeline

### 📝 Configuration Required
Admins must configure:
1. Enable `enable_multimodal_vision` in admin settings
2. Select vision-capable model (e.g., `gpt-4o`, `gpt-4-vision-preview`)
3. For Video Indexer: Configure managed identity and ARM resource details
4. Enable `enable_extract_meta_data` to surface metadata citations

---

This PR significantly enhances the application's multi-modal capabilities, providing users with richer context from images and documents while maintaining efficient token usage and robust security practices.

* Conversation Management Features (#532)

New Features
1. Pin Conversations
Users can pin important conversations to keep them at the top of the list
Pinned conversations display a pin icon (📌) in the conversation header and details modal
Pin icon appears before the conversation title
Bulk pin/unpin operations available in multi-select mode
Pinned conversations always appear first, sorted by most recent activity
2. Hide Conversations
Users can hide conversations to declutter their workspace without deleting them
Hidden conversations display an eye-slash icon (👁️‍🗨️) in the conversation header and details modal
Eye-slash icon appears next to the pin icon (if pinned)
Bulk hide/unhide operations available in multi-select mode
Toggle visibility of hidden conversations using the eye icon in the sidebar
3. Two-Tier Conversation Search
Quick Search (Sidebar)
Instant title-based filtering of conversations
Search icon in sidebar activates inline search input
Real-time filtering as you type
Clear button to reset search
Expand button to open advanced search modal
Advanced Search (Modal)
Full-text search across all message content
Multiple filter options:
Date range (from/to)
Chat type (personal/group/public)
Classifications (multi-select)
Has uploaded files
Has generated images
Pagination (20 results per page)
Message snippets with highlighted search terms (50 chars before/after match)
Click to navigate directly to specific messages
Search history tracking (last 20 searches)
Clickable search history to repeat searches
4. Message Highlighting & Navigation
Search results highlight matched text in yellow (amber in dark mode)
Smooth scroll animation to navigate to specific messages
Pulse animation draws attention to the target message
Highlights persist for 30 seconds before auto-clearing
Works across conversation switches
5. Multi-Select Mode
Select multiple conversations for bulk operations
Visual checkboxes appear when entering selection mode
Bulk actions available:
Pin/unpin selected conversations
Hide/unhide selected conversations
Delete selected conversations
Selection mode accessible from conversation dropdown menu
Auto-exit after 30 seconds of inactivity
6. Enhanced Conversation Details Modal
Displays pin icon if conversation is pinned
Displays eye-slash icon if conversation is hidden
Shows both icons at the top of the modal (next to title)
Status section shows visual badges for pinned/hidden state
Comprehensive metadata display
Technical Implementation
Frontend Changes
chat-conversations.js: Core conversation management, quick search, pin/hide functionality
chat-search-modal.js (NEW): Advanced search modal implementation
chat-sidebar-conversations.js: Sidebar search synchronization, hidden conversation handling
chat-messages.js: Message highlighting, smooth scroll, search highlight persistence
chat-conversation-details.js: Updated to show pin/hidden icons in modal
chats.css: Styles for search highlights and message pulse animations
HTML Templates: Added search modal, updated navigation icons
Backend Changes
route_backend_conversations.py:

/api/search_conversations - Full-text search with filters and pagination
/api/conversations/classifications - Get unique classification values
/api/user-settings/search-history - GET/POST/DELETE endpoints for search history
/api/conversations/{id}/pin - Toggle pin status
/api/conversations/{id}/hide - Toggle hide status
Bulk operations for pin/hide/delete
functions_settings.py: Search history management functions

* Message management (#553)

* added message masking

mask selected content of message or an entire message

* fixed citation border

* enabled streaming

* image gen with streaming

* added reasoning support

* added reasoning to agents

* agent support

* fixed key bug

* disable group create and fixed model fetch

* updated config

* fixed support for workspace search for streaming

* fix bug with sidebar update

* fixed gpt-5 vision processing bug

* metadata works with all messages now

* fixed debug_print bug

* added reasoning effort to agents and fixed agent validation

* fixed file metadata loading bug

* fixed llm streaming when working with group workspace data

* fixed cosmos container config error

* added delete message and fixed message threading

* retry bug fixes

* fixed message threading order

* moved message buttons to menu

* fixed bug for conversation history that included inactive threads

* added css styling for urls for dark mode

* fixed bug with newly created messages not showing metadata or deleting

* improved search times by 100x

* added token collect to messages supports models and agents

* added streaming for agents along with token collection

* added embedding token tracking

* added document creation/deletion and token tracking to activity log

* adding conversations to activity logs

* added activity log viewer with filters, search, and export

* added support for agents in edit and retry messages

* Configure Application from AZD Up command (#548)

* Add Cosmos DB post-configuration script and update requirements - Initial POC

* post deploy configure services in cosmosdb

* refactor to prevent post deploy configuration + begin support of key based auth.

* Add additional parameter validation for creating entra app

* Refactor Bicep modules for improved authentication and key management

- Added keyVault-Secrets.bicep module for storing secrets in Key Vault.
- Modified keyVault.bicep to remove enterprise app client secret handling and commented out managed identity role assignments.
- Removed openAI-existing.bicep and refactored openAI.bicep to handle model deployments dynamically.
- Added setPermissions.bicep for managing role assignments for various resources.
- Updated postconfig.py to reflect changes in environment variable handling for authentication type.

* Refactor Bicep modules to conditionally add settings based on authentication type and enable resource declarations for services

* initial support for VideoIndexer service

* Refactor Bicep modules to enhance VideoIndexer service integration and update diagnostic settings configurations

* move from using chainguard-dev builder image to python slim image.

* Updates to support post deployment app config

* Add post-deployment permissions script for CosmosDB and update authentication type handling

* fix typo in enhanced citation deployment config

* Refactor Dockerfile to use Python 3.13-slim and streamline build process

* restart web application after deployment settings applied

* remove setting for disableLocalAuth

* update to latest version of bicep deployment

* remove dead code

* code cleanup / formatting

* removed unnecessary content from readme.md

* fix token scope for commericial search service

* set permission correctly for lookup of openAI models

* fixes required to configure search with managed identity

* Adds Azure Billing Plugin in Community Customizations (#546)

* add crude keyvault base impl

* upd actions for MAG

* add settings to fix

* upd secret naming convention

* upd auth types to include conn string/basic(un/pw)

* fix method name

* add get agent helper

* add ui trigger word and get agent helper

* upd function imports

* upd agents call

* add desc of plugins

* fix for admin modal loading

* upd default agent handling

* rmv unneeded file

* rmv extra imp statements

* add new cosmos container script

* upd instructions for consistency of code

* adds safe calls for akv functions

* adds akv to personal agents

* fix for user agents boot issue

* fix global set

* upd azure function plugin to super init

* upd to clean imports

* add keyvault to global actions loading

* add plugin loading docs

* rmv secret leak via logging

* rmv displaying of token in logs

* fix not loading global actions for personal agents

* rmv unsupported characters from logging

* fix chat links in dark mode

* chg order of css for links in dark mode

* fix chat color

* add default plugin print logging

* rmv default check for nonsql plugins

* upd requirements

* add keyvault and dynamic addsetting ui

* fix for agents/plugins with invalid akv chars

* add imp to appins logging

* add security tab UI + key vault UI

* add keyvault settings

* fix for copilot findings.

* fix for resaving plugin without changing secret

* init azure billing plugin

* add app settings cache

* upd to azure billing plugin

* upd to msgraph plugin

* init community customizations

* add module

* add key vault config modal

* add logging and functions to math

* rmv extra telemetry, add appcache

* upd billing plugin

* add/upd key vault, admin settings, agents, max tokens

* Remove abp for pr

* disable static logging for development

* rmv dup import

* add note on pass

* added notes

* rmv dup decl

* add semicolon

* rmv unused variable add agent name to log

* add actions migration back in

* add notes and copilot fixes

* add abp back in

* upd abp/seperate graph from qu…
@paullizer
Record main ancestry for v0.250.001 release flow
8ab56b9
@paullizer
Record Staging ancestry for v0.250.001 release flow
b15b7e4
@paullizer
Merge pull request #929 from microsoft/fix/repair-release-ancestry-v0…
b49e5d6 
….250.001-v2

Repair v0.250.001 release ancestry with merge commits
@paullizer
Merge pull request #930 from microsoft/Development
189e829 
Sync Development to Staging with repaired release ancestry
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@github-code-quality github-code-quality[bot] github-code-quality[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@paullizer @Bionic711 @eldong @Xeelee33 @SteveCInVA @chadpalmer @clarked-msft @nadoylemsft @vivche