Few exceptions and solutions while implementing new drill functions

The post talks about two exceptions I came across while trying to implement new Drill Functions. I would keep updating the list as i come across new ones.

  • ExpressionParsingException
  • SqlValidationException

ExpressionParsingException:

This error pops up when we try to invoke a new/unimplemented function which Drill is not aware of. The exception can be solved by implementing the new Drill Function. Here is a short post on implementing new Math Functions for Drill. Contribution guide – Drill Math Functions.

0: jdbc:drill:schema=parquet-local> SELECT ABS(2.3) FROM "sample-data/region.parquet";
Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while running query.[error_id: "6e482ea4-9578-4a33-a268-077cbec014f9"
endpoint {
 address: "localhost"
 user_port: 31010
 bit_port: 31011
}
error_type: 0
message: "Failure while setting up Foreman. < ExpressionParsingException:[ Unable to find function definition for function named \'abs\' ]"
]
java.lang.RuntimeException: org.apache.drill.exec.rpc.RpcException: Remote failure while running query.[error_id: "6e482ea4-9578-4a33-a268-077cbec014f9"
endpoint {
 address: "localhost"
 user_port: 31010
 bit_port: 31011
}
error_type: 0
message: "Failure while setting up Foreman. < ExpressionParsingException:[ Unable to find function definition for function named \'abs\' ]"
]
 at org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:61)
 at net.hydromatic.optiq.runtime.ObjectEnumeratorCursor.next(ObjectEnumeratorCursor.java:44)
 at net.hydromatic.optiq.jdbc.OptiqResultSet.next(OptiqResultSet.java:162)
 at sqlline.SqlLine$BufferedRows.<init>(SqlLine.java:2499)
 at sqlline.SqlLine.print(SqlLine.java:1886)
 at sqlline.SqlLine$Commands.execute(SqlLine.java:3835)
 at sqlline.SqlLine$Commands.sql(SqlLine.java:3738)
 at sqlline.SqlLine.dispatch(SqlLine.java:882)
 at sqlline.SqlLine.begin(SqlLine.java:717)
 at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:460)
 at sqlline.SqlLine.main(SqlLine.java:443)
Caused by: org.apache.drill.exec.rpc.RpcException: Remote failure while running query.[error_id: "6e482ea4-9578-4a33-a268-077cbec014f9"
endpoint {
 address: "localhost"
 user_port: 31010
 bit_port: 31011
}
error_type: 0
message: "Failure while setting up Foreman. < ExpressionParsingException:[ Unable to find function definition for function named \'abs\' ]"
]
 at org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)
 at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79)
 at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48)
 at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33)
 at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142)
 at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127)
 at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
 at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)
 at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)
 at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
 at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)
 at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)
 at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173)
 at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)
 at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)
 at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)
 at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100)
 at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497)
 at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465)
 at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359)
 at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
 at java.lang.Thread.run(Thread.java:724)

 

 SqlValidationException:

This is a more serious exception that I came across. This Exception arises from Optiq Parser. Drill uses Optiq for parsing its SQL queries and there is a list of allowed keywords from optiq. Once Optiq passes the query then it reaches to Drill and looks for the function implementations.

So we cannot use the SQLLINE prompt to test any new function that is not registered with Optiq. The only way to test the new functions is to write a LogicalPlan plan directly for drill and test it by using a test case. The test case would use the reference interpreter to bypass Optiq’s validation and we can move ahead testing/implementing newer Drill Functions.

Look into RunSimplePlan java file for reference:
/drill-ref/src/test/java/org/apache/drill/exec/ref/RunSimplePlan.java

 

0: jdbc:drill:schema=parquet-local> SELECT ROUND(2.3) FROM "sample-data/region.parquet";
Sep 25, 2013 11:15:02 AM org.eigenbase.sql.validate.SqlValidatorException <init>
SEVERE: org.eigenbase.sql.validate.SqlValidatorException: No match found for function signature ROUND(<NUMERIC>)
Sep 25, 2013 11:15:02 AM org.eigenbase.util.EigenbaseException <init>
SEVERE: org.eigenbase.util.EigenbaseContextException: From line 1, column 8 to line 1, column 17
org.eigenbase.util.EigenbaseContextException: From line 1, column 8 to line 1, column 17
 at org.eigenbase.resource.EigenbaseResource$_Def12.ex(EigenbaseResource.java:1026)
 at org.eigenbase.sql.SqlUtil.newContextException(SqlUtil.java:739)
 at org.eigenbase.sql.SqlUtil.newContextException(SqlUtil.java:726)
 at org.eigenbase.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:3830)
 at org.eigenbase.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1594)
 at org.eigenbase.sql.SqlFunction.deriveType(SqlFunction.java:312)
 at org.eigenbase.sql.SqlFunction.deriveType(SqlFunction.java:222)
 at org.eigenbase.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:4238)
 at org.eigenbase.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:4220)
 at org.eigenbase.sql.SqlCall.accept(SqlCall.java:187)
 at org.eigenbase.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1502)
 at org.eigenbase.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1484)
 at org.eigenbase.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:435)
 at org.eigenbase.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:3290)
 at org.eigenbase.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3026)
 at org.eigenbase.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:69)
 at org.eigenbase.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:90)
 at org.eigenbase.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:802)
 at org.eigenbase.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:790)
 at org.eigenbase.sql.SqlSelect.validate(SqlSelect.java:154)
 at org.eigenbase.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:753)
 at org.eigenbase.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:444)
 at org.eigenbase.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:445)
 at net.hydromatic.optiq.prepare.Prepare.prepareSql(Prepare.java:160)
 at net.hydromatic.optiq.prepare.Prepare.prepareSql(Prepare.java:129)
 at net.hydromatic.optiq.prepare.OptiqPrepareImpl.prepare2_(OptiqPrepareImpl.java:255)
 at net.hydromatic.optiq.prepare.OptiqPrepareImpl.prepare_(OptiqPrepareImpl.java:195)
 at net.hydromatic.optiq.prepare.OptiqPrepareImpl.prepareSql(OptiqPrepareImpl.java:168)
 at net.hydromatic.optiq.jdbc.OptiqStatement.parseQuery(OptiqStatement.java:402)
 at net.hydromatic.optiq.jdbc.OptiqStatement.execute(OptiqStatement.java:192)
 at sqlline.SqlLine$Commands.execute(SqlLine.java:3825)
 at sqlline.SqlLine$Commands.sql(SqlLine.java:3738)
 at sqlline.SqlLine.dispatch(SqlLine.java:882)
 at sqlline.SqlLine.begin(SqlLine.java:717)
 at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:460)
 at sqlline.SqlLine.main(SqlLine.java:443)
Caused by: org.eigenbase.sql.validate.SqlValidatorException: No match found for function signature ROUND(<NUMERIC>)
 at org.eigenbase.resource.EigenbaseResource$_Def9.ex(EigenbaseResource.java:963)
 ... 32 more

 

Yash Sharma is a Big Data & Machine Learning Engineer, A newbie OpenSource contributor, Plays guitar and enjoys teaching as part time hobby.
Talk to Yash about Distributed Systems and Data platform designs.

Leave a Reply

Your email address will not be published. Required fields are marked *