Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ISSUE] Dependency Conflicts (log4j) & ClassCastException while using JDBC driver #394

Open
amarchigurupati opened this issue Jan 2, 2025 · 1 comment
Open

[ISSUE] Dependency Conflicts (log4j) & ClassCastException while using JDBC driver #394

amarchigurupati opened this issue Jan 2, 2025 · 1 comment

Comments

@amarchigurupati
Copy link

amarchigurupati commented Jan 2, 2025 •
edited
Loading

Description
Using JDBC driver to interact with tables on Databricks catalog.
Connection is successful. But there are Exceptions in logs while accessing the result set. However, I can read results.

Perhaps this is the result of incompatibility between my log4j version and a version that's shaded in the databricks-jdbc JAR.

ERROR StatusLogger Unable to create Lookup for bundle
java.lang.ClassCastException: class org.apache.logging.log4j.core.lookup.ResourceBundleLookup
at java.base/java.lang.Class.asSubclass(Class.java:4102)

ERROR StatusLogger Unable to create Lookup for date
java.lang.ClassCastException: class org.apache.logging.log4j.core.lookup.DateLookup
at java.base/java.lang.Class.asSubclass(Class.java:4102)

mvn dependency:tree -Dverbose -Dincludes=org.apache.logging.log4j
Result of above command:

[INFO] com.xyz:jar:0.0.1-SNAPSHOT
[INFO] +- com.xyz-auth:jar:0.2.2:compile
[INFO] |  \- com.xyz.util:jar:0.2.2:compile
[INFO] |     \- org.apache.logging.log4j:log4j-core:jar:2.24.1:compile (version managed from 2.24.2)
[INFO] |        \- org.apache.logging.log4j:log4j-api:jar:2.24.1:compile (version managed from 2.24.1)
[INFO] \- org.springframework.boot:spring-boot-starter-actuator:jar:3.4.0:compile
[INFO]    \- org.springframework.boot:spring-boot-starter:jar:3.4.0:compile (version managed from 3.4.0; scope not updated to compile)
[INFO]       \- org.springframework.boot:spring-boot-starter-logging:jar:3.4.0:compile (version managed from 3.4.0)
[INFO]          \- org.apache.logging.log4j:log4j-to-slf4j:jar:2.24.1:compile (version managed from 2.24.1)
[INFO]             \- (org.apache.logging.log4j:log4j-api:jar:2.24.1:compile - version managed from 2.24.1; omitted for duplicate)

Reproduction
Nothing specific. We are using spring boot and databricks jdbc driver. Running simple queries to interact with catalog tables and iterating over result set.

Expected behavior
No Exception from databricks seen in the logs.

Is it a regression?
I don't no. we are using databricks-jdbc 2.6.40-patch-1

Debug Logs

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
ERROR StatusLogger Unable to create Lookup for bundle
java.lang.ClassCastException: class org.apache.logging.log4j.core.lookup.ResourceBundleLookup
at java.base/java.lang.Class.asSubclass(Class.java:4102)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:84)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:105)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.config.AbstractConfiguration.<init>(AbstractConfiguration.java:135)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.config.NullConfiguration.<init>(NullConfiguration.java:32)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.LoggerContext.<clinit>(LoggerContext.java:74)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.selector.ClassLoaderContextSelector.createContext(ClassLoaderContextSelector.java:254)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getDefault(ClassLoaderContextSelector.java:266)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:146)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:123)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:230)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:47)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.LogManager.getContext(LogManager.java:176)
at com.databricks.client.jdbc42.internal.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:60)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:47)
at com.databricks.client.jdbc42.internal.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:33)
at com.databricks.client.jdbc42.internal.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
at com.databricks.client.jdbc42.internal.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at com.databricks.client.jdbc42.internal.apache.arrow.memory.BaseAllocator.<clinit>(BaseAllocator.java:46)
at com.databricks.client.spark.arrow.ArrowBuffer.<init>(Unknown Source)
at com.databricks.client.spark.jdbc.SparkJDBC42Driver.createArrowBuffer(Unknown Source)
at com.databricks.client.hivecommon.api.HS2Client.makeRowsetBuffer(Unknown Source)
at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.<init>(Unknown Source)
at com.databricks.client.hivecommon.dataengine.HiveJDBCResultSet.moveToNextRow(Unknown Source)
at com.databricks.client.jdbc.common.BaseForwardResultSet.next(Unknown Source)


ERROR StatusLogger Unable to create Lookup for ctx
java.lang.ClassCastException: class org.apache.logging.log4j.core.lookup.ContextMapLookup
at java.base/java.lang.Class.asSubclass(Class.java:4102)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:84)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:105)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.config.AbstractConfiguration.<init>(AbstractConfiguration.java:135)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.config.NullConfiguration.<init>(NullConfiguration.java:32)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.LoggerContext.<clinit>(LoggerContext.java:74)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.selector.ClassLoaderContextSelector.createContext(ClassLoaderContextSelector.java:254)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getDefault(ClassLoaderContextSelector.java:266)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:146)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:123)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:230)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:47)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.LogManager.getContext(LogManager.java:176)
at com.databricks.client.jdbc42.internal.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:60)
at com.databricks.client.jdbc42.internal.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:47)
at com.databricks.client.jdbc42.internal.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:33)
at com.databricks.client.jdbc42.internal.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
at com.databricks.client.jdbc42.internal.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at com.databricks.client.jdbc42.internal.apache.arrow.memory.BaseAllocator.<clinit>(BaseAllocator.java:46)
at com.databricks.client.spark.arrow.ArrowBuffer.<init>(Unknown Source)
at com.databricks.client.spark.jdbc.SparkJDBC42Driver.createArrowBuffer(Unknown Source)
at com.databricks.client.hivecommon.api.HS2Client.makeRowsetBuffer(Unknown Source)
at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.<init>(Unknown Source)
at com.databricks.client.hivecommon.dataengine.HiveJDBCResultSet.moveToNextRow(Unknown Source)
at com.databricks.client.jdbc.common.BaseForwardResultSet.next(Unknown Source)


....
java.lang.ClassCastException: class org.apache.logging.log4j.core.lookup.DateLookup

java.lang.ClassCastException: class org.apache.logging.log4j.core.lookup.EnvironmentLookup

...

Other Information

  • OS: [e.g. macOS]
  • Version: [e.g. 0.1.0]

Additional context
As obvious, excluding log4j dependencies from databricks-jdbc didn't work. Since these dependencies maybe have repackaged to a different location/package.

<!-- JDBC drivers -->
        <dependency>
            <groupId>com.databricks</groupId>
            <artifactId>databricks-jdbc</artifactId>
            <version>2.6.40-patch-1</version>
            <exclusions>
                <exclusion>
                    <groupId>org.apache.logging.log4j</groupId>
                    <artifactId>log4j-core</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.apache.logging.log4j</groupId>
                    <artifactId>log4j-api</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.apache.logging.log4j</groupId>
                    <artifactId>log4j-to-slf4j</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
@amarchigurupati
Copy link
Author

Workaround
Adding EnableArrow=0 to the JDBC URL is not showing these Exceptions or conflicts.

But I'm interested to know how to get-rid of these exceptions with enabling apache arrow optimizations?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@amarchigurupati