open source data platform for data teams
includes:
- data portal
- generic CRUD application to edit database tables
- adhoc tool to executes predefined SQL queries
- ... and more to come
cd frontend
deploy_frontend.bat dev|test|prod
cd backend
deploy_backend.bat dev|test|prodThe backend is configured via environment variables, typically placed in a .env file. A custom config file path can be set via PLAINBI_BACKEND_CONFIG.
| Variable | Default | Description |
|---|---|---|
PLAINBI_BACKEND_CONFIG |
— | Path to a custom .env config file. If set, this file is loaded instead of the default search paths (.env, ~/.env, /etc/plainbi.env). |
PLAINBI_REPOSITORY |
(required) | SQLAlchemy connection string for the plainbi repository database (SQLite). The backend exits on startup if this is missing. |
PLAINBI_DATABASE |
— | Default datasource connection string. Additional datasources are managed via the repository. |
| Variable | Default | Description |
|---|---|---|
PLAINBI_BACKEND_HOST |
0.0.0.0 |
Host address the API server binds to. |
PLAINBI_BACKEND_PORT |
3001 |
TCP port the API server listens on. |
| Variable | Default | Description |
|---|---|---|
PLAINBI_BACKEND_LOGFILE |
— | Path to a log file. If not set, logs go to stdout only. |
PLAINBI_BACKEND_LOG_DEBUG |
false |
Set to true to enable DEBUG-level logging. |
PLAINBI_VERBOSE |
— | Integer ≥ 1 enables DEBUG logging and increases internal verbosity. Takes priority over PLAINBI_BACKEND_LOG_DEBUG. |
plainbi caches table metadata (column names, data types, primary keys) in memory to avoid repeated queries to the database's information schema on every operation.
| Variable | Default | Description |
|---|---|---|
PLAINBI_METADATA_CACHE |
true |
Set to false to disable the metadata cache entirely. |
PLAINBI_METADATA_CACHE_TTL |
300 |
Cache lifetime in seconds. After expiry the metadata is re-fetched from the database. |
For Snowflake, metadata is fetched via SHOW COLUMNS / SHOW PRIMARY KEYS (~75ms) instead of information_schema (~880ms). If the SHOW commands fail for any reason, plainbi falls back to information_schema automatically.
Snowflake uses JWT-based authentication (private key). JWT tokens expire after ~60 minutes. plainbi handles this transparently:
- A connection pool (
pool_size=5,pool_recycle=1800s) is used for performance — connections are reused within the recycle window. - If a query fails with a JWT expiry error (Snowflake error
390144or similar), the connection is immediately invalidated, a fresh connection with a new JWT token is obtained, and the query is retried once — without surfacing an error to the user. CLIENT_SESSION_KEEP_ALIVE=TRUEin the connection string keeps Snowflake sessions alive during inactivity but does not prevent JWT token expiry. The retry mechanism above handles that.
The Snowflake connection string is stored in the plainbi_datasource table and typically looks like:
snowflake://<user>@<account>/?warehouse=<warehouse>&database=<db>&schema=PUBLIC&role=<role>&authenticator=SNOWFLAKE_JWT&CLIENT_SESSION_KEEP_ALIVE=TRUE&private_key=<key>
| Variable | Default | Description |
|---|---|---|
PLAINBI_BACKEND_DATE_FORMAT |
— | Python strftime format string for date values, e.g. %d.%m.%Y. If not set, the database default is used. |
PLAINBI_BACKEND_DATETIME_FORMAT |
— | Python strftime format string for datetime values, e.g. %d.%m.%Y %H:%M. If not set, the database default is used. |
All API calls (CRUD, adhoc, metadata, login) are logged to the plainbi_audit table with:
username— user who made the requestt— timestampurl— request URLid— resource id (e.g. adhoc ID for adhoc requests)request_method— HTTP method (GET, POST, etc.)request_body— request body (truncated)status— 'ok' or 'error'error_msg— error message if status='error' (up to 2000 chars)duration_ms— request execution time in milliseconds
A view plainbi_audit_adhoc filters audit entries for adhoc requests and joins them with user and adhoc metadata:
SELECT user_name, username, datum, adhoc_id, adhoc_name, ausgabe_format, status, duration_ms, error_msg
FROM plainbi_audit_adhoc
ORDER BY datum DESC;Migration for existing installations: Run backend/migration_audit_status.sql to add the new columns to existing plainbi_audit tables (ALTER TABLE statements per database type).
If LDAP_HOST is set, plainbi uses LDAP for user authentication instead of local users.
| Variable | Default | Description |
|---|---|---|
LDAP_HOST |
— | LDAP server hostname. Setting this activates LDAP authentication. |
LDAP_PORT |
— | LDAP server port (e.g. 389). |
LDAP_BIND_USER_DN |
— | Distinguished name (DN) of the bind user used to search the directory. |
LDAP_BIND_USER_PASSWORD |
— | Password of the bind user. |
LDAP_BASE_DN |
— | Base DN for user search (e.g. ou=users,dc=example,dc=com). |
LDAP_SEARCH_EXPR |
— | Custom LDAP search filter. {username} is substituted with the login username. Default: (&(cn={username})). |
| Variable | Default | Description |
|---|---|---|
PLAINBI_SSO_APPLIKATION |
— | Display name of the SSO application. |
PLAINBI_SSO_APPLICATION_ID |
— | Azure AD application (client) ID. |
PLAINBI_SSO_TENANTID |
— | Azure AD tenant ID. |
PLAINBI_SSO_CLIENT_SECRET |
— | Azure AD client secret. |
PLAINBI_SSO_AUTHORITY |
— | Authority endpoint, e.g. https://login.microsoftonline.com/<tenant-id>. |
PLAINBI_SSO_REDIRECT_PATH |
— | Redirect path after successful SSO login, e.g. /auth/callback. |
Used for password-reset and notification features.
| Variable | Default | Description |
|---|---|---|
SMTP_SERVER |
— | SMTP server hostname, e.g. smtp.gmail.com. |
SMTP_PORT |
— | SMTP server port, e.g. 587. |
SMTP_USER |
— | SMTP login username / sender address. |
SMTP_PASSWORD |
— | SMTP password. If not set, unauthenticated relay is attempted. |
A CRUD application is defined as code. Following syntax is possible:
{
"pages":[
{
"id":"1",
"name":"<Page name>",
"alias":"<page_alias>",
"allowed_actions":[
"update", "create", "delete", "duplicate", "export_dsdb", "export_excel", "view_calendar"
],
"pk_columns":["<primary_key_column>, ..."],
"table":"<table>",
"versioned": "true",
"table_for_list":"<table>",
"sequence":"<sequence>",
"hide_in_navigation":"true",
"show_breadcrumb":"true",
"parent_page": {"alias":"<alias_of_parent_page>","name":"<Label of parent page>"},
"user_column":"<column>",
"order": [
{
"column_name": "<technical_column_name>",
"direction": "asc|desc"
},
{
"column_name": "<technical_column_name>"
},
...
],
"conditional_row_formats": [
{
"column_name":"<technical_column_name>",
"operator":"eq|neq|gt|ge|lt|le",
"value":"<value>",
"style":{
"background-color":"rgb(..., ..., ...)",
"...":"..."
}
},
...
],
"detail_pages": [
{
"alias":"<alias_of_detail_page>",
"fk_column":"<fk_column_in_detail_table>",
"label":"<Tab label>"
},
...
],
"external_actions": [
{
"type": "call_rest_api",
"id": "1",
"label": "<Label to show on button>",
"tooltip": "<just a tooltip to show on button>",
"method": "POST|GET",
"contenttype": "application/json",
"url": "<any endpoint url>",
"body": "<any payload e.g plain, JSON, etc.",
"token": "<optional Bearer token, supports ${username} and ${column_name}>",
"wait_repeat_in_ms": "<ms>",
"position": "detail|summary"
},
{
"type": "call_stored_procedure",
"id": "2",
"label": "<Label to show on button>",
"tooltip": "<just a tooltip to show on button>",
"name": "<name of stored procedure in database including schema e.g. <schema>.<stored_procedure>" ,
"body": "{\"@param1\":\"value1\", ...}",
"wait_repeat_in_ms": "<ms>",
"position": "detail|summary"
}
],
"table_columns":[
{
"column_name":"<technical_column_name>",
"column_label":"<Label to show>",
"datatype":"text",
"ui":"textinput",
"lookup":"<lookup_alias>",
"multiple":"true",
"tooltip":"<just a tooltip>",
"editable":"false|true",
"required":"false|true",
"showdetailsonly":"true",
"showsummaryonly":"true",
"default_value": "<value>",
"calendar_field": "<column_name>"
},
{
...
},
{
...
}
]
},
{
"id":"2",
"name":"<Page name 2>",
"alias":"<page_alias>",
...
},
{
...
}
]
}Just any numerical value: 1,2,3,...
Has to be unique for all pages.
A nice label for the page :)
A alias (without spaces and special characters) that is mainly used to identify the page and also used in the URL, so the page can also be adressed directly
Treats a table as a versioned table (SCD2) and is able to generate appropriate records.
Following fields are required in such a table:
last_changed_by varchar(100)
valid_from_dt datetime
invalid_from_dt datetime
last_changed_dt datetime
is_deleted char(1) -- Y/N
is_latest_period char(1) -- Y/N
is_current_and_active char(1) -- Y/NThe last_changed_by field is filled with the username. So you can track who creates, edits or deletes a record.
leave empty array if no actions allowed or select between these options (in any combination)
actions on row level
updatecreatedeleteduplicate(same ascreate, but prefills the form with the data of the selected row)export_dsdb(custom functionality to export a plainbi application or lookup as .dsdb file for datasqill used in DevOps scenario (git versioning)
actions on page level
export_excel(enable download of the displayed table as Excel file (.xlsx))view_calendar(enable switching between table view and a calendar view - if used, then you have to match the fields of the table to the calendar specific fields: id, title, subtitle, start, end, color, url - see "table_columns" for more information)
Used for editing and deleting a record - composite key possible: ["primary_key_column1", "primary_key_column2", ...]
The most important option. Defines the table to be used for all CRUD operations (create, read, update, delete).
Has to be the fully qualified name: <database>.schema.tablename>
optional: used as an alternative only for the tabular view of a page — can be a different table or a view, e.g. a view that already JOINs lookup tables so that labels are shown instead of raw IDs in the list. This is the recommended approach for displaying human-readable values in the tabular view when a column uses ui: lookup.
optional: sequence to use for the primary key column when adding a new record
optional: hides the page in the side navigation
optional:
- show_breadcrumb: shows a breadcrumb above the page
- parent_page: refers to the parent page (used for the breadcrumb to show a navigation e.g. "parent page" > "current page") - array of "alias" and "name" of the parent page
optional: defines which column contains the username and tells plainbi to write the username of the logged in user into this column, when creating or editing a record (insert / update)
optional: defines a predefined / default sort order of the data. An array is given with one or more table columns and a sort direction (ascending or descending). The sort direction is optional and defaults to "ascending"
optional: allow to format a row conditionally depending on the value of a column
Array of one or more table columns.
Here following options are possible:
- column_name: just the technical column name
- column_label: a nice label to show
- datatype: allowed values:
text,number,date,datetime,boolean - ui: allowed values
textinput(standard single-line text input)email(single-line text input, semantically typed as email)numberinput(only allows numerical values)textarea(multi-line plain text input)textarea_markdown(multi-line text input with live Markdown preview)textarea_sql(Monaco code editor for SQL — with syntax highlighting and expand/collapse; read-only wheneditableisfalse)textarea_json(Monaco code editor for JSON — editable, with syntax highlighting, format and validate buttons, expand/collapse)textarea_base64(textarea that displays a base64 encoded image preview underneath)datepicker(date picker, stores value as YYYY-MM-DD)datetimepicker(date + time picker, stores value as YYYY-MM-DD HH:mm)switch(toggle switch, good for boolean values 1/0 or true/false)label(shows the value as plain text — not editable)hidden(field is not shown in the form at all)lookup(dropdown with server-side search and infinite scroll — refers to a lookup; shows 50 entries at a time, filtered live as you type)lookupn(same as lookup, but also allows entering new values not in the list)password(text input with masked characters)password_nomem(same as password, but prevents browser from saving/autofilling the value)html(only used in tabular view — renders raw HTML)modal_json_to_table(only used in tabular view — renders a JSON array as a nested table)
- lookup: refers to a lookup with its alias - only used for "ui":"lookup" or "ui":"lookupn"
- editable: allowed values
false,true - required: allowed values
false,true - multiple: optional: allows multiple values to be selected - only used for "ui":"lookup"
- tooltip: optional: shows a question icon next to the field name and shows a tooltip when hovering - good to use for explanations
- showdetailsonly: allowed value
true: optional: show this field only in detail view (modal dialog) - showsummaryonly: allowed value
true: optional: show this field only in tabular view - default_value: set a default value when creating a new data entry
- calendar_field: optional: matches the column to a calendar specific field- allowed values:
id,title,subtitle,start,end,color,url- only used for action "view_calendar"
optional: Enables Master/Detail editing. When a record is opened for editing, the modal shows tabs — "Stammdaten" for the master form plus one tab per detail page.
Each detail page must be defined as a regular page in the same pages array (typically with "hide_in_navigation": true). The detail_pages array just references it by alias and defines the FK mapping:
"detail_pages": [
{
"alias": "<alias of the detail page>",
"fk_column": "<FK column name in the detail table>",
"label": "<Tab label to display>",
"static_values": { "<column>": "<fixed value>", "...": "..." }
}
]alias: references an existing page in this application by its aliasfk_column: the column in the detail table that holds the FK value pointing to the master recordlabel: the tab label shown in the modalstatic_values: optional — additional fixed column values applied to both the list filter and new-record pre-fill. Useful for generic/shared detail tables (e.g. a comments table used by multiple entities).
When creating a new detail record from within the master modal, the FK column and any static_values columns are automatically pre-filled and set to read-only.
Using a generic table for multiple entities — a single comments table (entitaet, entitaet_nr, kommentar) can be reused across different master pages by combining fk_column with static_values:
"detail_pages": [
{
"alias": "kommentare",
"fk_column": "entitaet_nr",
"label": "Kommentare",
"static_values": { "entitaet": "Kunden" }
}
]This filters the detail list by entitaet_nr = <master PK> AND entitaet = 'Kunden' and pre-fills both columns when creating a new comment.
Example — master page adhoc with a detail page adhoc_berechtigungen:
{
"pages": [
{
"id": "1", "name": "Adhoc", "alias": "adhoc",
"table": "db.schema.adhoc", "pk_columns": ["adhoc_id"],
"allowed_actions": ["update", "create", "delete"],
"table_columns": [ ... ],
"detail_pages": [
{ "alias": "adhoc_berechtigungen", "fk_column": "adhoc_id", "label": "Berechtigungen" }
]
},
{
"id": "2", "name": "Adhoc Berechtigungen", "alias": "adhoc_berechtigungen",
"hide_in_navigation": true,
"table": "db.schema.adhoc_berechtigungen", "pk_columns": ["berechtigung_id"],
"allowed_actions": ["update", "create", "delete"],
"table_columns": [
{ "column_name": "berechtigung_id", "column_label": "ID", "ui": "numberinput", "editable": false },
{ "column_name": "adhoc_id", "column_label": "Adhoc ID", "ui": "numberinput", "editable": false },
{ "column_name": "user_name", "column_label": "Benutzer", "ui": "textinput", "editable": true }
]
}
]
}Allows the execution of external actions. This can defined per page and each action is offered as a button (next to the "New" button). Following actions are possible:
- call of a REST API (executed by the frontend)
- call of a stored procedure in the source database (executed by the backend) - only works for MS SQL Server at the moment
call of a REST API
The call only reacts on a positive or negative response and tries to get the error message in case of an error. But for a REST API call it is not possible to do a further call e.g. if you trigger an async process and want to wait/loop for a final status.
Here following options are available (and all are mandatory if not otherwise mentioned):
- type: allowed value
call_rest_api - id: a random unique number
- label: label to show on button
- method: only POST oder GET are supported
- contenttype: any contenttype e.g. "application/json"
- url: any endpoint url - be aware of CORS, because the frontend sends the request - if necessary add a routing (proxy path) to the webserver (e.g. Nginx)
- body:
- any payload e.g plain, JSON, etc.
- when using double-quote's then please escape them e.g. \"
- ${username} is substituted with the logged-in user
- ${<column_name>} is substituted with the value of the column of a dataset (only works when "position" is set to "detail")
- wait_repeat_in_ms: time to wait in milliseconds, before a repeat of the action is allowed. this prevents an action to be called to often from the user
- optional token: Bearer token for authorization. If set, the request includes
Authorization: Bearer <token>. TheBearerprefix is added automatically if not already present. Supports${username}and${<column_name>}substitution (column substitution only in detail/modal position). If omitted, no Authorization header is sent. - optional tooltip: a tooltip to show on the button
- optional position: detail (show in Modal) or summary (show on Page - is default)
call of a stored procedure
Calls the procedure and waits for it to finish with a success or error message.
- type: allowed value
call_stored_procedure - id: a random unique number
- label: label to show on button
- name: name of the stored procedure that is being called. fully qualified name with schema.
- body:
- used to hand over parameter values
- wait_repeat_in_ms: time to wait in milliseconds, before a repeat of the action is allowed. this prevents an action to be called to often from the user
- optional tooltip: a tooltip to show on the button
- optional position: detail (show in Modal) or summary (show on Page - is default)
Adhoc queries are predefined SQL statements that users can execute directly from the portal. They are managed in the Adhoc Konfiguration application and displayed on the home page for users with access.
Any value can be injected into the SQL at runtime using the $(placeholder_name) syntax:
SELECT *
FROM orders
WHERE customer_id = $(customer_id)
AND order_date >= $(date_from)
AND order_date <= $(date_to)The following global placeholders are always available without defining a parameter:
| Placeholder | Value |
|---|---|
$(APP_USER) |
Username of the currently logged-in user |
$(APP_USER_EMAIL) |
E-mail address of the currently logged-in user |
Parameters are defined per adhoc query in the Adhoc Konfiguration application under the Parameter tab. Each parameter corresponds to one $(placeholder_name) in the SQL.
| Field | Description |
|---|---|
name |
Display label shown to the user in the input form |
name_technical |
Must match the placeholder name in the SQL exactly (without $()) |
datatype |
Data type: text, number, date, datetime |
ui |
Input widget: textinput, numberinput, datepicker, lookup |
lookup |
Lookup alias — only used when ui is lookup (dropdown with values from a lookup table) |
default_value |
Pre-filled value when the form opens. The query runs immediately with this value. |
required |
Whether the field must be filled before executing |
order_by_default |
Default sort order for the result table, e.g. column_name asc or column_name:desc. Applied to both the HTML view and Excel/CSV export. A URL parameter ?order_by=... takes precedence. |
When an adhoc has one or more parameters, the user sees a collapsible filter panel above the results. The query runs automatically on page load only if all required parameters have a default value or are supplied via URL. If any required parameter has no value, the filter panel is shown and the user must fill it in before executing. Results are paginated server-side (50 rows per page by default; the user can change the page size via the pagination control).
Clicking Ausführen re-runs the query with the current parameter values. Active column filters and sort order are preserved — they are not reset when re-executing.
Note on string and date parameters: Parameter values are substituted as plain strings into the SQL without automatic quoting. For string and date values, add quotes around the placeholder in your SQL:
'$(date_from)'not$(date_from).
Example — SQL with two parameters:
SELECT order_id, customer_name, amount
FROM orders
WHERE status = $(status)
AND created_by = $(APP_USER)
AND order_date >= $(date_from)Corresponding parameter definitions:
| name | name_technical | ui | default_value | required |
|---|---|---|---|---|
| Status | status | lookup | open | true |
| Datum ab | date_from | datepicker | false |
Every application and page can be adressed by a well formed and human friendly URL. This is defined by the aliases.
Application:
https://<server>/apps/<app_alias>
Page within application:
https://<server>/apps/<app_alias>/<page_alias>
Also by using parameters you can achieve following...
Directly edit a record within a page:
https://<server>/apps/<app_alias>/<page_alias>/<id_of_a_record> # if record has 1 pk (primary key field) ... # todo: composite key
Filter the tabular view:
https://<server>/apps/<app_alias>/<page_alias>?<field>=<value> # field = technical column name
https://<server>/apps/<app_alias>/<page_alias>?<field>=<value>&<field>=<value>&... # also more than 1 field is possible
URL filters are applied as exact matches and shown as blue tags above the table. Clicking the × on a tag removes the filter from the URL. Multiple URL filters are combined with AND.
The tabular view of a CRUD page includes a toolbar with search, column settings and a reset button.
A search box is shown on the left of the toolbar. Typing searches all columns with a 600 ms debounce. Pressing Enter searches immediately. Clicking the × clears the search and refreshes the table.
The Spalten button (⏸ icon) opens a drawer where you can:
- Show/hide columns — toggle visibility per column
- Reorder columns — drag the grip handle to change the column order
- Resize columns — drag the right edge of any column header to resize it
All changes are saved to localStorage per page and restored on the next visit.
Clicking a column header sorts by that column (ascending → descending → unsorted). The active sort is saved to localStorage and restored when the page is reopened.
A Zurücksetzen button (orange) appears in the toolbar whenever the current state differs from the default — i.e. when any of the following are active: search text, column filters, sort order, hidden columns, reordered columns, or custom column widths. Clicking it resets all of the above back to defaults and clears the saved state for the current page.
All state (search, sort, column filters, column settings) is saved separately per page, keyed by URL path and table name.
Each column in the tabular view has a filter icon (funnel) in the header. Clicking it opens a dropdown with:
- A search input to type a filter value freely
- A scrollable list of distinct values from the database (server-side, 50 at a time with infinite scroll)
- Click a value from the list to apply it immediately, or type and click Übernehmen
- Zurücksetzen clears the filter for that column
Active column filters appear as tags above the table alongside any URL-based filters. Multiple column filters are combined with AND and also apply to Excel/CSV exports. Alle löschen removes all column filters at once.
Column filters use a LIKE '%value%' match (case-insensitive). URL filters use an exact match.
Adhoc result tables also support server-side sorting: clicking a column header cycles through ascending / descending / unsorted. Multiple columns can be sorted simultaneously. The current sort order is included when exporting to Excel or CSV.
Adhoc queries can be pre-filled and triggered directly via URL. The name_technical of each parameter is used as the URL query parameter name.
Pre-fill the filter form and execute automatically:
https://<server>/adhoc/<alias>?<name_technical>=<value>&<name_technical>=<value>
The filter panel opens pre-filled with the given values and the query runs immediately. The panel is shown collapsed so the result is visible right away.
Download as Excel or CSV directly:
https://<server>/adhoc/<alias>?format=XLSX&<name_technical>=<value>
https://<server>/adhoc/<alias>?format=CSV&<name_technical>=<value>
Auto-run without showing the filter form at all (drill-down link):
https://<server>/adhoc/<alias>?autorun=1&<name_technical>=<value>
With autorun=1 the filter panel is hidden and the query executes immediately. For Excel/CSV downloads, the browser is redirected back to the home page after the download starts. Parameters not supplied via URL fall back to their configured default_value.
When downloading an adhoc result as Excel (.xlsx), the file contains:
- daten sheet — the query result, respecting active parameter filters, column filters and current sort order
- info sheet — metadata with the following rows:
- Erstellt am — creation timestamp (seconds precision, e.g.
2025-05-29 14:30:00) - Adhoc — adhoc name and ID in parentheses, e.g.
Umsatz nach Region (42) - Beschreibung — adhoc description
- Filter (if parameters are active) — one row per parameter with its display label and value; for lookup parameters the human-readable display value is shown instead of the technical key
- Spaltenfilter (if column filters are active) — one row per active column filter showing column name and filter value
- Erstellt am — creation timestamp (seconds precision, e.g.
- sql sheet (hidden) — the executed SQL statement for traceability
The filename follows the pattern Adhoc_<id>_<name>_<timestamp>.xlsx, e.g. Adhoc_42_Umsatz_nach_Region_2025-05-29T14:30:00.xlsx. The adhoc name is truncated to 50 characters and special characters are replaced with underscores.