Repository navigation
Column name replacement caused by @Column annotation produces invalid SQL for property named 'in'聽#2684
Description
Rewriting this into an issue since we were advised that it'll be more useful than a Google Group post 馃槈 :
Expected behavior
Properties named 'in' can be used in Java code by specifying a different column name using the @column annotation.
Actual behavior
Properties named 'in' with a @column annotation produce SQL syntax errors since the SQL keyword 'in' is also being replaced by the column name specified for the property.
For example, if we have a property named 'in' with a @column(name="inputName") annotation and the correctly created SQL would be:
select t0.id, t0.status, t0.name, t0.inputName, t0.smallnote, t0.anniversary, t0.cretime, t0.updtime, t0.version, t0.billing_address_id, t0.shipping_address_id from o_customer t0 where t0.name in (?,?)
the SQL-"in" of the where clause also mistakenly gets replaced by the column name, producing
select t0.id, t0.status, t0.name, t0.inputName, t0.smallnote, t0.anniversary, t0.cretime, t0.updtime, t0.version, t0.billing_address_id, t0.shipping_address_id from o_customer t0 where t0.name t0.inputName (?,?)
which leads to a SQLException due to invalid syntax.
Steps to reproduce
Add a property named 'in' with an arbitrary column name by using @column(name = "...") to any entity, for example Customer in ebean-test:
@Size(max = 40)
@Column(name = "inputName")
String in;And start all tests, which will produce several failures, for example EqlParserTest.where_in_when_namedParams_withNoWhitespace:
Error: Tests run: 83, Failures: 0, Errors: 7, Skipped: 3, Time elapsed: 0.311 s <<< FAILURE! - in io.ebean.xtest.internal.server.grammer.EqlParserTest
Error: io.ebean.xtest.internal.server.grammer.EqlParserTest.where_in_when_namedParams_withNoWhitespace Time elapsed: 0.003 s <<< ERROR!
javax.persistence.PersistenceException:
Query threw SQLException:Syntax error in SQL statement "select t0.id, t0.status, t0.name, t0.inputName, t0.smallnote, t0.anniversary, t0.cretime, t0.updtime, t0.version, t0.billing_address_id, t0.shipping_address_id from o_customer t0 where t0.name [*]t0.inputName (?,?)"; SQL statement:
select t0.id, t0.status, t0.name, t0.inputName, t0.smallnote, t0.anniversary, t0.cretime, t0.updtime, t0.version, t0.billing_address_id, t0.shipping_address_id from o_customer t0 where t0.name t0.inputName (?,?) [42000-212] Bind values:[null] Query was:select t0.id, t0.status, t0.name, t0.inputName, t0.smallnote, t0.anniversary, t0.cretime, t0.updtime, t0.version, t0.billing_address_id, t0.shipping_address_id from o_customer t0 where t0.name t0.inputName (?,?)
Activity
rbygrave commentedon May 4, 2022
The question becomes ... why don't you change that logical from property from
into something else?focbenz commentedon May 5, 2022
We did ... the property's maiden name ist now
inParams.So we do not have an impediment but wanted to point this out to the community and get your opinion on if this should be treated as a bug and in that case deliver a test case for it.
As Tegmango pointed out, he tried to use the Google Groups mailing list first (https://groups.google.com/g/ebean/c/zW-Ng_nhHNo), but got no response. So sorry for spamming the typical Ebean issues watcher 馃槆
Feel free to close this one if you think the effort is not worth it to make
@Column(name = "somethingElseThanIn")work in that special SQL keyword case. We can also add some documentation near the@Columnannotation that with specific SQL reserved keywords likeinit might be better to just avoid them.rbygrave commentedon May 5, 2022
Ah, I'm not sure why I didn't see that. If someone could add any sort of reply onto that google group post ... to check that I see it in my email that would be good.
Yup, cool.
There isn't enough information yet. This issue mentions
EqlParserTestso maybe it is only hit via EQL parsing? If so maybe that is worth looking at, but there are other places where logical to physical parsing mapping occurs - so really we need more specific failing tests.focAsbrand commentedon May 5, 2022
I have compiled a list of failing tests due to SQL syntax errors caused by adding the optional property in the Customer class (extracted out of FOCONIS@fb0994b):
An example stack trace:
The stack traces of the failures all share that the exception is produced during the find/update operation of CQueryEngine, i.e.
for the findList/findMap operations or
for the update operations.
rPraml commentedon May 5, 2022
I tried briefly to narrow down the problem:
CQueryPredicates::parsePropertiesToDbColumns tries to convert properties to db columns
is invoked with
whereExprSql = "id in (?,?,?,?,?)"The return value is
${}id ${}inputName (?,?,?,?,?)This sounds logical, as the deployParser is not intelligent enough to detect that 'in' is an operand here and no property.
(corner case:
in in (?,?,?,?,?)the firstinis a property, the second an operand)I think a fix (if there even is an easy one) should be located somewhere here: https://github.com/ebean-orm/ebean/blob/master/ebean-core/src/main/java/io/ebeaninternal/server/deploy/DeployParser.java#L68
There are already some special handlings for table name implemented (see skipWordConvert())
Perhaps one can say that once a property has been converted, the next word must be an operand and must not be converted.
I'll think about it again
rPraml commentedon May 5, 2022
@focbenz @Tegmango AFAIR from our last discussion, there was an existing interface
And you had an
Entity implements InterFaceright?So, it was obvious to name the properties
inandoutI've made some experiments, if I can modify the DeployParser to detect, if
inis a property or an operator:I thougt it would make sense to treat every word after an replacement as operator. This fixed most of the test, but things like
sum(status * name)did not work, as the*isn't handled in the word loop but infindWordStart()Then I checked, if the word is really an operator. This would work, as long you do not have raw SQL like
concat(out, in)(Note:concat(in, out)would work.I think this requires some rewrite of the DeployParser, where I really do not know, if it is worth, because there could be many situations, where you can trick the parser
1=1 in (true)I thought with
@ColumnNameyou are safe, if you use a reserved property name. (We often usekeyandvalue)But for now, I would say, reserved names should be avoided whenever possible, especially if they are operators.
@rbygrave what do you think, should this be fixed or is a restriction of ebean
rbygrave commentedon May 5, 2022
I think the question is, is there a design change that would skip CQueryPredicates::parsePropertiesToDbColumns for the non-raw sql case. (Approximately, change such that expressions that are not
raw()don't get the "raw sql parsing")That is, produce
"${}id in (?,?,?,?,?)"and skip the parsing for this expression altogether. Note: the${}prefix here means "root level so replace${}with the root level table alias which is generallyt0.${customer.billingAddress}citymeans replace${customer.billingAddress}with the table alias for the join tocustomer.billingAddress(for example, that might bet1).Currently, "raw expression parsing" occurs on all the expressions - not just
raw(). That is, while we generally writequery.where().eq("customer.name", "rob")we can actually doquery.where().eq("any_sql_function_we_like(customer.name)", "rob")... theeqexpression actually gets the same "raw sql parsing" applied to it.That is, we are encouraged via documentation to use
raw()for cases like this via:query.where().raw("any_sql_function_we_like(customer.name) = ?", "rob")... but actually the raw parsing also occurs foreq,inetc - well, all expressions in fact. That is, CQueryPredicates::parsePropertiesToDbColumns operates on the whole where clause, not "just the raw parts".So that thinking heads towards ... maybe ebean should not do that and instead be stricter such that only
rawexpressions get the full "raw parsing".so, maybe the problem becomes more obvious if we look at the example:
query.where().raw("any_sql_function_we_like(in) in (?,?,?)", "a", "b, "c")... because
inexists twice as both "logical property name" AND "sql operator in". BUT, if we ever hit this case we can instead NOT use the logical property nameinand use the physicalt0.input_nameinstead like:query.where().raw("any_sql_function_we_like(t0.input_name) in (?,?,?)", "a", "b, "c")... but today, the raw parser would still replace that
inoperator with${}input_name... so it wouldn't quite work. We'd need some ability to turn off the raw parsing of this expression -> "ebean, please don't parse this because I have a logical bean property that is also the same name as a sql operator I'm using" (in, exists etc)Hmmm.
rbygrave commentedon May 5, 2022
Hmmm, I wonder if its just a case of
current -> instead produce
id in (?,?)->${}id in (?,?)in in (?,?)->${}in in (?,?)customer.name = ?->${customer}name = ?customer.billingAddress.city in (?,?)->${customer.billingAddress}city in (?,?)raw()expressions.raw("any_sql_function_we_like(customer.name) in (?,?,?)", "a", "b, "c")produces ...
$ebean[any_sql_function_we_like(customer.name) in (?,?,?)]ebean$... and the parser only works on content between
$ebean[and]ebean$and outputs for that section of the contentany_sql_function_we_like(${customer}name) in (?,?,?).rawUnparsed()expression:.rawUnparsed("any_sql_function_we_like(t0.input_name) in (?,?,?)", "a", "b, "c")that produces content without the
$ebean[and]ebean$... so skips parsing. We are safe to haveinas a logical bean property when using.rawUnparsed()- it's not going to try and convertintot0.input_namehere.rbygrave commentedon Jun 26, 2022
Ok, I'm going to transfer this into Discusssions -> Backlog
The outstanding action, is to investigate if we can and should improve the parsing as described in the last post.