[mkurz]

• Fixes [2.7.x][5.0.x] Evolution fails for auto-generated script contains create procedures script #166
• Fixes Evolution can't be run due to "delimiter $$" DDL parsing bug - Play 3, Ebean 8 #496

Some of the statements in Ebean's DDL contain semicolons:

• the stored procedures that Ebean creates on MySQL, MariaDB and SQL Server
• the triggers for @History (Postgres, MySQL)
• the blocks with which Oracle and DB2 drop sequences in the drop-all DDL
• column comments with a semicolon

Ebean separates such statements with its own conventions, like delimiter $$ or GO. Play's evolutions split on every semicolon instead, so the generated 1.sql failed to apply, e.g. with "near 'delimiter $$'" on MySQL and MariaDB, "Incorrect syntax near 'GO'" on SQL Server or "Unterminated dollar quote" on Postgres.

Play now has a !split-semicolon directive for evolution scripts:

• Add a !split-semicolon directive to evolution scripts playframework#14346

so Play is updated to 3.1.0-M10-3968a052-SNAPSHOT, which has it.

Play Ebean now splits the DDL with Ebean's own DdlParser, which Ebean's DdlRunner uses as well, and writes each statement with a semicolon between comments, like this (from MySQL, shortened):

-- !split-semicolon: never
CREATE PROCEDURE usp_ebean_drop_column(IN p_table_name VARCHAR(255), IN p_column_name VARCHAR(255))
BEGIN
CALL usp_ebean_drop_foreign_keys(p_table_name, p_column_name);
SET @sql = CONCAT('ALTER TABLE `', p_table_name, '` DROP COLUMN `', p_column_name, '`');
PREPARE stmt FROM @sql;
EXECUTE stmt;
END
-- !split-semicolon: always
;

• If Play already runs Ebean's statements from the DDL, the DDL stays exactly as it is. So a script only changes where Play would have split Ebean's statements differently.
• The DDL header (Ebean's ddl.header property), which can contain anything, always stays as it is.

The docs explain this. They also say that once a production database uses the generated script, the script should stop being regenerated: remove the -- Created by Ebean DDL comment, or turn off generating the script of that Ebean server with play.ebean.db.<server>.generateEvolutionsScripts (or of all Ebean servers with play.ebean.generateEvolutionsScripts), especially before upgrading to Play Ebean 9, whose scripts differ where the DDL has such statements. Upgrading Play alone doesn't regenerate anything.

Tests:

• EbeanEvolutionScriptTest generates the DDL offline for all of Ebean's platforms, with a sequence, @History and a comment with semicolons. It checks that Play runs the statements Ebean would run, for the Ups and the Downs. Without this change, this fails on MySQL, MariaDB, SQL Server, Postgres, HANA, NuoDB, YugabyteDB, Oracle and DB2. It also covers the DDL header and DDL that stays as it is.
• EbeanEvolutionScriptDatabasesTest applies the generated script with Play's evolutions, uses the models (including @History versions and a stored procedure), and reverts the script, twice. It runs on H2 and SQLite always, and on Postgres, MySQL, MariaDB, SQL Server and Oracle when PLAY_EBEAN_TEST_<DB>_URL, _USER and _PASSWORD (or _PASSWORD_FILE) are set.
• Locally, all of these passed on Postgres 17, MySQL 8.4, MariaDB 11.4, SQL Server 2022 and Oracle 23ai Free, and testFull passed on Scala 2.13.x, 3.3.x, 3.9.x and 3.next. CI only runs the H2 and SQLite variants.

How to run the database tests (also in the README, with Docker commands for each database):

• Set these variables for each database: PLAY_EBEAN_TEST_<DB>_URL, _USER, and _PASSWORD or _PASSWORD_FILE, where <DB> is POSTGRES, MYSQL, MARIADB, SQLSERVER or ORACLE. A database without them is skipped.
• Use an empty, disposable database. The test creates and drops model and history tables. It also creates Ebean helper procedures, which may remain afterward.

docker run -d --name play-ebean-test-postgres -e POSTGRES_PASSWORD=test -e POSTGRES_DB=play_ebean_test -p 127.0.0.1:15432:5432 postgres:17
export PLAY_EBEAN_TEST_POSTGRES_URL='jdbc:postgresql://127.0.0.1:15432/play_ebean_test' PLAY_EBEAN_TEST_POSTGRES_USER=postgres PLAY_EBEAN_TEST_POSTGRES_PASSWORD=test
sbt --server 'core/testOnly play.db.ebean.EbeanEvolutionScriptDatabasesTest'

With --server, sbt starts on its own and sees the variables. Without it, sbt can connect to an sbt server that's already running, which doesn't see them.

Before/after examples of the generated 1.sql, created offline from the test models. "Before" is what Play Ebean writes without this change, "after" what it writes with it:

• Where Play already splits the DDL like Ebean, e.g. on H2 without such statements, the script stays byte-identical.
• Ups and Downs are handled separately. Here, only Oracle's Downs change.
• In a rewritten part, Ebean's comment lines between statements (like -- apply changes) are gone, since Ebean's parser drops them too, and statements are separated by blank lines.

MySQL / MariaDB: stored procedures with delimiter $$

--- before/1.sql
+++ after/1.sql
@@ -1,71 +1,71 @@
 -- Created by Ebean DDL
 -- To stop Ebean DDL generation, remove this comment (both lines) and start using Evolutions
 
 -- !Ups
 
--- init script create procs
--- Inital script to create stored procedures etc for mysql platform
 DROP PROCEDURE IF EXISTS usp_ebean_drop_foreign_keys;
 
-delimiter $$
+-- !split-semicolon: never
 --
 -- PROCEDURE: usp_ebean_drop_foreign_keys TABLE, COLUMN
 -- deletes all constraints and foreign keys referring to TABLE.COLUMN
 --
 CREATE PROCEDURE usp_ebean_drop_foreign_keys(IN p_table_name VARCHAR(255), IN p_column_name VARCHAR(255))
 -- play-ebean-start
 BEGIN
 DECLARE done INT DEFAULT FALSE;
 DECLARE c_fk_name CHAR(255);
 DECLARE curs CURSOR FOR SELECT CONSTRAINT_NAME from information_schema.KEY_COLUMN_USAGE
 WHERE TABLE_SCHEMA = DATABASE() and TABLE_NAME = p_table_name and COLUMN_NAME = p_column_name
 AND REFERENCED_TABLE_NAME IS NOT NULL;
 DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
 
 OPEN curs;
 
 read_loop: LOOP
 FETCH curs INTO c_fk_name;
 IF done THEN
 LEAVE read_loop;
 END IF;
 SET @sql = CONCAT('ALTER TABLE `', p_table_name, '` DROP FOREIGN KEY ', c_fk_name);
 PREPARE stmt FROM @sql;
 EXECUTE stmt;
 END LOOP;
 
 CLOSE curs;
 -- play-ebean-end
 END
-$$
+-- !split-semicolon: always
+;
 
 DROP PROCEDURE IF EXISTS usp_ebean_drop_column;
 
-delimiter $$
+-- !split-semicolon: never
 --
 -- PROCEDURE: usp_ebean_drop_column TABLE, COLUMN
 -- deletes the column and ensures that all indices and constraints are dropped first
 --
 CREATE PROCEDURE usp_ebean_drop_column(IN p_table_name VARCHAR(255), IN p_column_name VARCHAR(255))
 -- play-ebean-start
 BEGIN
 CALL usp_ebean_drop_foreign_keys(p_table_name, p_column_name);
 SET @sql = CONCAT('ALTER TABLE `', p_table_name, '` DROP COLUMN `', p_column_name, '`');
 PREPARE stmt FROM @sql;
 EXECUTE stmt;
 -- play-ebean-end
 END
-$$
--- apply changes
+-- !split-semicolon: always
+;
+
 create table author (
   id                            bigint auto_increment not null,
   name                          varchar(255),
   constraint pk_author primary key (id)
 );
 
 
 -- !Downs
 
 -- drop all
 drop table if exists author;
 

SQL Server: GO and stored procedures

--- before/1.sql
+++ after/1.sql
@@ -1,150 +1,138 @@
 -- Created by Ebean DDL
 -- To stop Ebean DDL generation, remove this comment (both lines) and start using Evolutions
 
 -- !Ups
 
--- init script create procs
--- Initial script to create stored procedures etc for sqlserver platform
+if not exists (select name  from sys.types where name = 'ebean_bigint_tvp') create type ebean_bigint_tvp as table (c1 bigint);
 
--- create table-value-parameters
-if not exists (select name  from sys.types where name = 'ebean_bigint_tvp') create type ebean_bigint_tvp as table (c1 bigint)
-GO
-if not exists (select name  from sys.types where name = 'ebean_float_tvp') create type ebean_float_tvp as table (c1 float)
-GO
-if not exists (select name  from sys.types where name = 'ebean_bit_tvp') create type ebean_bit_tvp as table (c1 bit)
-GO
-if not exists (select name  from sys.types where name = 'ebean_date_tvp') create type ebean_date_tvp as table (c1 date)
-GO
-if not exists (select name  from sys.types where name = 'ebean_time_tvp') create type ebean_time_tvp as table (c1 time)
-GO
-if not exists (select name  from sys.types where name = 'ebean_uniqueidentifier_tvp') create type ebean_uniqueidentifier_tvp as table (c1 uniqueidentifier)
-GO
-if not exists (select name  from sys.types where name = 'ebean_nvarchar_tvp') create type ebean_nvarchar_tvp as table (c1 nvarchar(max))
-GO
-
---
--- PROCEDURE: usp_ebean_drop_indices TABLE, COLUMN
--- deletes all indices referring to TABLE.COLUMN
---
+if not exists (select name  from sys.types where name = 'ebean_float_tvp') create type ebean_float_tvp as table (c1 float);
+
+if not exists (select name  from sys.types where name = 'ebean_bit_tvp') create type ebean_bit_tvp as table (c1 bit);
+
+if not exists (select name  from sys.types where name = 'ebean_date_tvp') create type ebean_date_tvp as table (c1 date);
+
+if not exists (select name  from sys.types where name = 'ebean_time_tvp') create type ebean_time_tvp as table (c1 time);
+
+if not exists (select name  from sys.types where name = 'ebean_uniqueidentifier_tvp') create type ebean_uniqueidentifier_tvp as table (c1 uniqueidentifier);
+
+if not exists (select name  from sys.types where name = 'ebean_nvarchar_tvp') create type ebean_nvarchar_tvp as table (c1 nvarchar(max));
+
+-- !split-semicolon: never
 CREATE OR ALTER PROCEDURE usp_ebean_drop_indices @tableName nvarchar(255), @columnName nvarchar(255)
 AS
 -- play-ebean-start
 SET NOCOUNT ON
 declare @sql nvarchar(1000)
 declare @indexName nvarchar(255)
 BEGIN
   DECLARE index_cursor CURSOR FOR SELECT i.name from sys.indexes i
     join sys.index_columns ic on ic.object_id = i.object_id and ic.index_id = i.index_id
     join sys.columns c on c.object_id = ic.object_id and c.column_id = ic.column_id
     where i.object_id = OBJECT_ID(@tableName) AND c.name = @columnName;
   OPEN index_cursor
   FETCH NEXT FROM index_cursor INTO @indexName
   WHILE @@FETCH_STATUS = 0
     BEGIN
       set @sql = 'drop index ' + @indexName + ' on [' + @tableName + ']';
       EXECUTE(@sql);
 
       FETCH NEXT FROM index_cursor INTO @indexName
     END;
   CLOSE index_cursor;
   DEALLOCATE index_cursor;
 END
 -- play-ebean-end
-GO
+-- !split-semicolon: always
+;
 
---
--- PROCEDURE: usp_ebean_drop_default_constraint TABLE, COLUMN
--- deletes the default constraint, which has a random name
---
+-- !split-semicolon: never
 CREATE OR ALTER PROCEDURE usp_ebean_drop_default_constraint @tableName nvarchar(255), @columnName nvarchar(255)
 AS
 -- play-ebean-start
 SET NOCOUNT ON
 declare @tmp nvarchar(1000)
 BEGIN
   select @tmp = t1.name from sys.default_constraints t1
     join sys.columns t2 on t1.object_id = t2.default_object_id
     where t1.parent_object_id = OBJECT_ID(@tableName) and t2.name = @columnName;
 
   if @tmp is not null EXEC('alter table [' + @tableName +'] drop constraint ' + @tmp);
 END
 -- play-ebean-end
-GO
+-- !split-semicolon: always
+;
 
---
--- PROCEDURE: usp_ebean_drop_constraints TABLE, COLUMN
--- deletes constraints and foreign keys refering to TABLE.COLUMN
---
+-- !split-semicolon: never
 CREATE OR ALTER PROCEDURE usp_ebean_drop_constraints @tableName nvarchar(255), @columnName nvarchar(255)
 AS
 -- play-ebean-start
 SET NOCOUNT ON
 declare @sql nvarchar(1000)
 declare @constraintName nvarchar(255)
 BEGIN
   DECLARE name_cursor CURSOR FOR
   SELECT cc.name from sys.check_constraints cc
     join sys.columns c on c.object_id = cc.parent_object_id and c.column_id = cc.parent_column_id
     where parent_object_id = OBJECT_ID(@tableName) AND c.name = @columnName
   UNION SELECT fk.name from sys.foreign_keys fk
     join sys.foreign_key_columns fkc on fkc.constraint_object_id = fk.object_id
       and  fkc.parent_object_id = fk.parent_object_id
     join sys.columns c on c.object_id = fkc.parent_object_id and c.column_id = fkc.parent_column_id
     where fkc.parent_object_id = OBJECT_ID(@tableName) AND c.name = @columnName;
 
   OPEN name_cursor
   FETCH NEXT FROM name_cursor INTO @constraintName
   WHILE @@FETCH_STATUS = 0
     BEGIN
       set @sql = 'alter table [' + @tableName + '] drop constraint ' + @constraintName;
       EXECUTE(@sql);
 
       FETCH NEXT FROM name_cursor INTO @constraintName
     END;
   CLOSE name_cursor;
   DEALLOCATE name_cursor;
 END
 -- play-ebean-end
-GO
+-- !split-semicolon: always
+;
 
---
--- PROCEDURE: usp_ebean_drop_column TABLE, COLUMN
--- deletes the column annd ensures that all indices and constraints are dropped first
---
+-- !split-semicolon: never
 CREATE OR ALTER PROCEDURE usp_ebean_drop_column @tableName nvarchar(255), @columnName nvarchar(255)
 AS
 -- play-ebean-start
 SET NOCOUNT ON
 declare @sql nvarchar(1000)
 BEGIN
   EXEC usp_ebean_drop_indices @tableName, @columnName;
   EXEC usp_ebean_drop_default_constraint @tableName, @columnName;
   EXEC usp_ebean_drop_constraints @tableName, @columnName;
 
   set @sql = 'alter table [' + @tableName + '] drop column [' + @columnName + ']';
   EXECUTE(@sql);
 END
 -- play-ebean-end
-GO
--- apply changes
+-- !split-semicolon: always
+;
+
 create table author (
   id                            bigint not null,
   name                          nvarchar(255),
   constraint pk_author primary key (id)
 );
+
 create sequence author_seq as bigint start with 1 increment by 50;
 
 
 -- !Downs
 
 -- drop all
 if exists (select name  from sys.types where name = 'ebean_bigint_tvp') drop type ebean_bigint_tvp;
 if exists (select name  from sys.types where name = 'ebean_float_tvp') drop type ebean_float_tvp;
 if exists (select name  from sys.types where name = 'ebean_bit_tvp') drop type ebean_bit_tvp;
 if exists (select name  from sys.types where name = 'ebean_date_tvp') drop type ebean_date_tvp;
 if exists (select name  from sys.types where name = 'ebean_time_tvp') drop type ebean_time_tvp;
 if exists (select name  from sys.types where name = 'ebean_uniqueidentifier_tvp') drop type ebean_uniqueidentifier_tvp;
 if exists (select name  from sys.types where name = 'ebean_nvarchar_tvp') drop type ebean_nvarchar_tvp;
 IF OBJECT_ID('author', 'U') IS NOT NULL drop table author;
 IF OBJECT_ID('author_seq', 'SO') IS NOT NULL drop sequence author_seq;
 

Postgres: @History trigger function with $$

--- before/1.sql
+++ after/1.sql
@@ -1,71 +1,73 @@
 -- Created by Ebean DDL
 -- To stop Ebean DDL generation, remove this comment (both lines) and start using Evolutions
 
 -- !Ups
 
--- apply changes
 create table author (
   id                            bigint generated by default as identity not null,
   name                          varchar,
   constraint pk_author primary key (id)
 );
 
 create table book (
   id                            bigint generated by default as identity not null,
   author_id                     bigint,
   title                         varchar,
   constraint pk_book primary key (id)
 );
 
--- apply alter tables
 alter table book add column if not exists sys_period tstzrange not null default tstzrange(current_timestamp, null);
--- apply post alter
+
 create table book_history(like book);
+
 create view book_with_history as select * from book union all select * from book_history;
+
+-- !split-semicolon: never
 create or replace function book_history_version() returns trigger as $$
 -- play-ebean-start
 declare
   lowerTs timestamptz;
   upperTs timestamptz;
 begin
   lowerTs = lower(OLD.sys_period);
   upperTs = greatest(lowerTs + '1 microsecond',current_timestamp);
   if (TG_OP = 'UPDATE') then
     insert into book_history (sys_period,id, title, author_id) values (tstzrange(lowerTs,upperTs), OLD.id, OLD.title, OLD.author_id);
     NEW.sys_period = tstzrange(upperTs,null);
     return new;
   elsif (TG_OP = 'DELETE') then
     insert into book_history (sys_period,id, title, author_id) values (tstzrange(lowerTs,upperTs), OLD.id, OLD.title, OLD.author_id);
     return old;
   end if;
 end;
 -- play-ebean-end
-$$ LANGUAGE plpgsql;
+$$ LANGUAGE plpgsql
+-- !split-semicolon: always
+;
 
 create trigger book_history_upd
   before update or delete on book
   for each row execute procedure book_history_version();
 
-
--- foreign keys and indices
 create index ix_book_author_id on book (author_id);
+
 alter table book add constraint fk_book_author_id foreign key (author_id) references author (id) on delete restrict on update restrict;
 
 
 -- !Downs
 
 -- drop all foreign keys
 alter table if exists book drop constraint if exists fk_book_author_id;
 drop index if exists ix_book_author_id;
 
 -- drop all
 drop table if exists author cascade;
 
 drop trigger if exists book_history_upd on book cascade;
 drop function if exists book_history_version();
 
 drop view book_with_history;
 drop table book_history;
 
 drop table if exists book cascade;
 

Oracle: PL/SQL block in the Downs that drops a sequence

--- before/1.sql
+++ after/1.sql
@@ -1,30 +1,35 @@
 -- Created by Ebean DDL
 -- To stop Ebean DDL generation, remove this comment (both lines) and start using Evolutions
 
 -- !Ups
 
--- apply changes
 create table publisher (
   id                            number(19) not null,
   name                          varchar2(255),
   constraint pk_publisher primary key (id)
 );
-comment on column publisher.name is 'Semicolons: ; and ;; stay as they are';
+
+-- !split-semicolon: never
+comment on column publisher.name is 'Semicolons: ; and ;; stay as they are'
+-- !split-semicolon: always
+;
+
 create sequence publisher_seq increment by 1;
 
 
 -- !Downs
 
--- drop all
 drop table publisher cascade constraints purge;
-delimiter $$
+
+-- !split-semicolon: never
 declare
   expected_error exception;
   pragma exception_init(expected_error, -2289);
 begin
   execute immediate 'drop sequence publisher_seq';
 exception
   when expected_error then null;
 end;
-$$;
+-- !split-semicolon: always
+;
 

Any platform (here H2): column comment with ; and ;;

--- before/1.sql
+++ after/1.sql
@@ -1,21 +1,25 @@
 -- Created by Ebean DDL
 -- To stop Ebean DDL generation, remove this comment (both lines) and start using Evolutions
 
 -- !Ups
 
--- apply changes
 create table publisher (
   id                            bigint not null,
   name                          varchar(255),
   constraint pk_publisher primary key (id)
 );
-comment on column publisher.name is 'Semicolons: ; and ;; stay as they are';
+
+-- !split-semicolon: never
+comment on column publisher.name is 'Semicolons: ; and ;; stay as they are'
+-- !split-semicolon: always
+;
+
 create sequence publisher_seq increment by 1;
 
 
 -- !Downs
 
 -- drop all
 drop table if exists publisher;
 drop sequence if exists publisher_seq;
 

To see them side by side: this gist has the same scripts, and its revisions show before and after in a split view.

[mergify]

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.