Repository navigation
[MDEP-640] Could not get correct module classpath with -pl option #1072
Description
Yuming Wang opened MDEP-640 and commented
build-classpath could not get correct classpath with -pl:
$ git clone https://github.com/wangyum/spark.git
$ cd spark && git checkout CARMEL-363
$ build/mvn -q versions:set -DnewVersion=CARMEL-363 -DgenerateBackupPoms=false -Phadoop-3.1
$ build/mvn -Phadoop-3.1 jar:jar jar:test-jar install:install clean
$ build/mvn -Phadoop-3.1 dependency:build-classpath -pl assemblyIt always add orc-core-1.5.4-nohive.jar and orc-mapreduce-1.5.4-nohive.jar to classpath:
RoaringBitmap-0.5.11.jar
accessors-smart-1.2.jar
activation-1.1.1.jar
aircompressor-0.10.jar
antlr4-runtime-4.7.1.jar
aopalliance-repackaged-2.4.0-b34.jar
arpack_combined_all-0.1.jar
arrow-format-0.12.0.jar
arrow-memory-0.12.0.jar
arrow-vector-0.12.0.jar
audience-annotations-0.5.0.jar
avro-1.8.2.jar
avro-ipc-1.8.2.jar
avro-mapred-1.8.2-hadoop2.jar
breeze-macros_2.12-0.13.2.jar
breeze_2.12-0.13.2.jar
chill-java-0.9.3.jar
chill_2.12-0.9.3.jar
commons-beanutils-1.9.3.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
commons-compiler-3.0.11.jar
commons-compress-1.9.jar
commons-configuration2-2.1.1.jar
commons-crypto-1.0.0.jar
commons-io-2.4.jar
commons-lang-2.6.jar
commons-lang3-3.8.1.jar
commons-math3-3.4.1.jar
commons-net-3.1.jar
commons-text-1.6.jar
compress-lzf-1.0.3.jar
core-1.1.2.jar
curator-client-2.13.0.jar
curator-framework-2.13.0.jar
curator-recipes-2.13.0.jar
dnsjava-2.1.7.jar
flatbuffers-java-1.9.0.jar
gson-2.2.4.jar
guava-14.0.1.jar
hadoop-annotations-3.2.0.jar
hadoop-auth-3.2.0.jar
hadoop-client-3.2.0.jar
hadoop-common-3.2.0.jar
hadoop-hdfs-client-3.2.0.jar
hadoop-mapreduce-client-common-3.2.0.jar
hadoop-mapreduce-client-core-3.2.0.jar
hadoop-mapreduce-client-jobclient-3.2.0.jar
hadoop-yarn-api-3.2.0.jar
hadoop-yarn-client-3.2.0.jar
hadoop-yarn-common-3.2.0.jar
hive-common-2.3.4.jar
hive-llap-client-2.3.4.jar
hive-llap-common-2.3.4.jar
hive-serde-2.3.4.jar
hive-service-rpc-2.3.4.jar
hive-shims-0.23-2.3.4.jar
hive-shims-2.3.4.jar
hive-shims-common-2.3.4.jar
hive-shims-scheduler-2.3.4.jar
hive-storage-api-2.6.0.jar
hk2-api-2.4.0-b34.jar
hk2-locator-2.4.0-b34.jar
hk2-utils-2.4.0-b34.jar
hppc-0.7.2.jar
htrace-core4-4.1.0-incubating.jar
httpclient-4.5.6.jar
httpcore-4.4.10.jar
ivy-2.4.0.jar
jackson-annotations-2.9.6.jar
jackson-core-2.9.6.jar
jackson-core-asl-1.9.13.jar
jackson-databind-2.9.6.jar
jackson-jaxrs-base-2.9.5.jar
jackson-jaxrs-json-provider-2.9.5.jar
jackson-mapper-asl-1.9.13.jar
jackson-module-jaxb-annotations-2.9.6.jar
jackson-module-paranamer-2.9.6.jar
jackson-module-scala_2.12-2.9.6.jar
janino-3.0.11.jar
javassist-3.18.1-GA.jar
javax.annotation-api-1.2.jar
javax.inject-2.4.0-b34.jar
javax.servlet-api-3.1.0.jar
javax.ws.rs-api-2.0.1.jar
jaxb-api-2.2.11.jar
jcip-annotations-1.0-1.jar
jcl-over-slf4j-1.7.16.jar
jersey-client-2.22.2.jar
jersey-common-2.22.2.jar
jersey-container-servlet-2.22.2.jar
jersey-container-servlet-core-2.22.2.jar
jersey-guava-2.22.2.jar
jersey-media-jaxb-2.22.2.jar
jersey-server-2.22.2.jar
jetty-webapp-9.3.24.v20180605.jar
jetty-xml-9.3.24.v20180605.jar
jline-2.14.6.jar
joda-time-2.9.3.jar
json-1.8.jar
json-smart-2.3.jar
json4s-ast_2.12-3.5.3.jar
json4s-core_2.12-3.5.3.jar
json4s-jackson_2.12-3.5.3.jar
json4s-scalap_2.12-3.5.3.jar
jsp-api-2.1.jar
jsr305-3.0.0.jar
jtransforms-2.4.0.jar
jul-to-slf4j-1.7.16.jar
kerb-admin-1.0.1.jar
kerb-client-1.0.1.jar
kerb-common-1.0.1.jar
kerb-core-1.0.1.jar
kerb-crypto-1.0.1.jar
kerb-identity-1.0.1.jar
kerb-server-1.0.1.jar
kerb-simplekdc-1.0.1.jar
kerb-util-1.0.1.jar
kerby-asn1-1.0.1.jar
kerby-config-1.0.1.jar
kerby-pkix-1.0.1.jar
kerby-util-1.0.1.jar
kerby-xdr-1.0.1.jar
kryo-shaded-4.0.2.jar
leveldbjni-all-1.8.jar
log4j-1.2.17.jar
lz4-java-1.5.0.jar
machinist_2.12-0.6.1.jar
macro-compat_2.12-1.1.1.jar
metrics-core-3.1.5.jar
metrics-graphite-3.1.5.jar
metrics-json-3.1.5.jar
metrics-jvm-3.1.5.jar
minlog-1.3.0.jar
netty-3.9.9.Final.jar
netty-all-4.1.30.Final.jar
nimbus-jose-jwt-4.41.1.jar
objenesis-2.5.1.jar
okhttp-2.7.5.jar
okio-1.6.0.jar
opencsv-2.3.jar
orc-core-1.5.4-nohive.jar
orc-core-1.5.4.jar
orc-mapreduce-1.5.4-nohive.jar
orc-shims-1.5.4.jar
oro-2.0.8.jar
osgi-resource-locator-1.0.1.jar
paranamer-2.8.jar
parquet-column-1.10.1.jar
parquet-common-1.10.1.jar
parquet-encoding-1.10.1.jar
parquet-format-2.4.0.jar
parquet-hadoop-1.10.1.jar
parquet-jackson-1.10.1.jar
protobuf-java-2.5.0.jar
py4j-0.10.8.1.jar
pyrolite-4.13.jar
re2j-1.1.jar
scala-compiler-2.12.8.jar
scala-library-2.12.8.jar
scala-parser-combinators_2.12-1.1.0.jar
scala-reflect-2.12.8.jar
scala-xml_2.12-1.0.5.jar
shapeless_2.12-2.3.2.jar
slf4j-api-1.7.16.jar
slf4j-log4j12-1.7.16.jar
snappy-java-1.1.7.1.jar
spark-catalyst_2.12-CARMEL-363.jar
spark-core_2.12-CARMEL-363.jar
spark-graphx_2.12-CARMEL-363.jar
spark-kvstore_2.12-CARMEL-363.jar
spark-launcher_2.12-CARMEL-363.jar
spark-mllib-local_2.12-CARMEL-363.jar
spark-mllib_2.12-CARMEL-363.jar
spark-network-common_2.12-CARMEL-363.jar
spark-network-shuffle_2.12-CARMEL-363.jar
spark-repl_2.12-CARMEL-363.jar
spark-sketch_2.12-CARMEL-363.jar
spark-sql_2.12-CARMEL-363.jar
spark-streaming_2.12-CARMEL-363.jar
spark-tags_2.12-CARMEL-363.jar
spark-unsafe_2.12-CARMEL-363.jar
spire-macros_2.12-0.13.0.jar
spire_2.12-0.13.0.jar
stax2-api-3.1.4.jar
stream-2.7.0.jar
token-provider-1.0.1.jar
univocity-parsers-2.7.3.jar
validation-api-1.1.0.Final.jar
woodstox-core-5.0.3.jar
xbean-asm7-shaded-4.12.jar
xz-1.5.jar
zookeeper-3.4.13.jar
zstd-jni-1.3.2-2.jar
But if remove -pl assembly and navigator to org.apache.spark:spark-assembly_2.12.
build/mvn -Phadoop-3.1 dependency:build-classpathThe classpath is:
RoaringBitmap-0.5.11.jar
accessors-smart-1.2.jar
activation-1.1.1.jar
aircompressor-0.10.jar
antlr4-runtime-4.7.1.jar
aopalliance-repackaged-2.4.0-b34.jar
arpack_combined_all-0.1.jar
arrow-format-0.12.0.jar
arrow-memory-0.12.0.jar
arrow-vector-0.12.0.jar
audience-annotations-0.5.0.jar
avro-1.8.2.jar
avro-ipc-1.8.2.jar
avro-mapred-1.8.2-hadoop2.jar
breeze-macros_2.12-0.13.2.jar
breeze_2.12-0.13.2.jar
chill-java-0.9.3.jar
chill_2.12-0.9.3.jar
commons-beanutils-1.9.3.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
commons-compiler-3.0.11.jar
commons-compress-1.9.jar
commons-configuration2-2.1.1.jar
commons-crypto-1.0.0.jar
commons-io-2.4.jar
commons-lang-2.6.jar
commons-lang3-3.8.1.jar
commons-math3-3.4.1.jar
commons-net-3.1.jar
commons-text-1.6.jar
compress-lzf-1.0.3.jar
core-1.1.2.jar
curator-client-2.13.0.jar
curator-framework-2.13.0.jar
curator-recipes-2.13.0.jar
dnsjava-2.1.7.jar
flatbuffers-java-1.9.0.jar
gson-2.2.4.jar
guava-14.0.1.jar
hadoop-annotations-3.2.0.jar
hadoop-auth-3.2.0.jar
hadoop-client-3.2.0.jar
hadoop-common-3.2.0.jar
hadoop-hdfs-client-3.2.0.jar
hadoop-mapreduce-client-common-3.2.0.jar
hadoop-mapreduce-client-core-3.2.0.jar
hadoop-mapreduce-client-jobclient-3.2.0.jar
hadoop-yarn-api-3.2.0.jar
hadoop-yarn-client-3.2.0.jar
hadoop-yarn-common-3.2.0.jar
hive-common-2.3.4.jar
hive-llap-client-2.3.4.jar
hive-llap-common-2.3.4.jar
hive-serde-2.3.4.jar
hive-service-rpc-2.3.4.jar
hive-shims-0.23-2.3.4.jar
hive-shims-2.3.4.jar
hive-shims-common-2.3.4.jar
hive-shims-scheduler-2.3.4.jar
hive-storage-api-2.6.0.jar
hk2-api-2.4.0-b34.jar
hk2-locator-2.4.0-b34.jar
hk2-utils-2.4.0-b34.jar
hppc-0.7.2.jar
htrace-core4-4.1.0-incubating.jar
httpclient-4.5.6.jar
httpcore-4.4.10.jar
ivy-2.4.0.jar
jackson-annotations-2.9.6.jar
jackson-core-2.9.6.jar
jackson-core-asl-1.9.13.jar
jackson-databind-2.9.6.jar
jackson-jaxrs-base-2.9.5.jar
jackson-jaxrs-json-provider-2.9.5.jar
jackson-mapper-asl-1.9.13.jar
jackson-module-jaxb-annotations-2.9.6.jar
jackson-module-paranamer-2.9.6.jar
jackson-module-scala_2.12-2.9.6.jar
janino-3.0.11.jar
javassist-3.18.1-GA.jar
javax.annotation-api-1.2.jar
javax.inject-2.4.0-b34.jar
javax.servlet-api-3.1.0.jar
javax.ws.rs-api-2.0.1.jar
jaxb-api-2.2.11.jar
jcip-annotations-1.0-1.jar
jcl-over-slf4j-1.7.16.jar
jersey-client-2.22.2.jar
jersey-common-2.22.2.jar
jersey-container-servlet-2.22.2.jar
jersey-container-servlet-core-2.22.2.jar
jersey-guava-2.22.2.jar
jersey-media-jaxb-2.22.2.jar
jersey-server-2.22.2.jar
jetty-webapp-9.3.24.v20180605.jar
jetty-xml-9.3.24.v20180605.jar
jline-2.14.6.jar
joda-time-2.9.3.jar
json-1.8.jar
json-smart-2.3.jar
json4s-ast_2.12-3.5.3.jar
json4s-core_2.12-3.5.3.jar
json4s-jackson_2.12-3.5.3.jar
json4s-scalap_2.12-3.5.3.jar
jsp-api-2.1.jar
jsr305-3.0.0.jar
jtransforms-2.4.0.jar
jul-to-slf4j-1.7.16.jar
kerb-admin-1.0.1.jar
kerb-client-1.0.1.jar
kerb-common-1.0.1.jar
kerb-core-1.0.1.jar
kerb-crypto-1.0.1.jar
kerb-identity-1.0.1.jar
kerb-server-1.0.1.jar
kerb-simplekdc-1.0.1.jar
kerb-util-1.0.1.jar
kerby-asn1-1.0.1.jar
kerby-config-1.0.1.jar
kerby-pkix-1.0.1.jar
kerby-util-1.0.1.jar
kerby-xdr-1.0.1.jar
kryo-shaded-4.0.2.jar
leveldbjni-all-1.8.jar
log4j-1.2.17.jar
lz4-java-1.5.0.jar
machinist_2.12-0.6.1.jar
macro-compat_2.12-1.1.1.jar
metrics-core-3.1.5.jar
metrics-graphite-3.1.5.jar
metrics-json-3.1.5.jar
metrics-jvm-3.1.5.jar
minlog-1.3.0.jar
netty-3.9.9.Final.jar
netty-all-4.1.30.Final.jar
nimbus-jose-jwt-4.41.1.jar
objenesis-2.5.1.jar
okhttp-2.7.5.jar
okio-1.6.0.jar
opencsv-2.3.jar
orc-core-1.5.4.jar
orc-mapreduce-1.5.4.jar
orc-shims-1.5.4.jar
oro-2.0.8.jar
osgi-resource-locator-1.0.1.jar
paranamer-2.8.jar
parquet-column-1.10.1.jar
parquet-common-1.10.1.jar
parquet-encoding-1.10.1.jar
parquet-format-2.4.0.jar
parquet-hadoop-1.10.1.jar
parquet-jackson-1.10.1.jar
protobuf-java-2.5.0.jar
py4j-0.10.8.1.jar
pyrolite-4.13.jar
re2j-1.1.jar
scala-compiler-2.12.8.jar
scala-library-2.12.8.jar
scala-parser-combinators_2.12-1.1.0.jar
scala-reflect-2.12.8.jar
scala-xml_2.12-1.0.5.jar
shapeless_2.12-2.3.2.jar
slf4j-api-1.7.16.jar
slf4j-log4j12-1.7.16.jar
snappy-java-1.1.7.1.jar
spark-catalyst_2.12-CARMEL-363.jar
spark-core_2.12-CARMEL-363.jar
spark-graphx_2.12-CARMEL-363.jar
spark-kvstore_2.12-CARMEL-363.jar
spark-launcher_2.12-CARMEL-363.jar
spark-mllib-local_2.12-CARMEL-363.jar
spark-mllib_2.12-CARMEL-363.jar
spark-network-common_2.12-CARMEL-363.jar
spark-network-shuffle_2.12-CARMEL-363.jar
spark-repl_2.12-CARMEL-363.jar
spark-sketch_2.12-CARMEL-363.jar
spark-sql_2.12-CARMEL-363.jar
spark-streaming_2.12-CARMEL-363.jar
spark-tags_2.12-CARMEL-363.jar
spark-unsafe_2.12-CARMEL-363.jar
spire-macros_2.12-0.13.0.jar
spire_2.12-0.13.0.jar
stax2-api-3.1.4.jar
stream-2.7.0.jar
token-provider-1.0.1.jar
univocity-parsers-2.7.3.jar
validation-api-1.1.0.Final.jar
woodstox-core-5.0.3.jar
xbean-asm7-shaded-4.12.jar
xz-1.5.jar
zookeeper-3.4.13.jar
zstd-jni-1.3.2-2.jar
Affects: 3.1.1
Activity
jira-importer commentedon Mar 17, 2019
Karl Heinz Marbaise commented
Can you please describe more in detail why you are calling the goals separately instead of using the life cycle?
slachiewicz commentedon Oct 7, 2026
Does it still happen on 3.11.0 if you run it through the lifecycle, for example
mvn -pl <module> -am package dependency:build-classpath? If so, please attach a small project that reproduces it; runningjar:jar,install:installandbuild-classpathas separate invocations skips the reactor resolution the goal relies on.