Skip to content

build(thirdparty): make RocksDB depend on Hadoop before building HDFS plugin - #2412

Open
empiredan wants to merge 1 commit into
apache:masterfrom
empiredan:rocksdb-depend-hadoop
Open

empiredan wants to merge 1 commit into
apache:masterfrom
empiredan:rocksdb-depend-hadoop

Conversation

@empiredan

Copy link
Copy Markdown
Contributor

Add an explicit thirdparty build dependency from RocksDB to Hadoop.

RocksDB builds the HDFS plugin on Linux, and that plugin includes hdfs.h from the
Hadoop native client output. Without a CMake ExternalProject dependency, RocksDB
and Hadoop can build in parallel, causing RocksDB to compile before Hadoop has copied
hdfs.h into thirdparty/output/include/hdfs.

This fixes intermittent thirdparty Docker build failures such as:
fatal error: hdfs.h: No such file or directory

Validation:

  • Verified the failure is caused by RocksDB HDFS plugin compiling before Hadoop native client output is available.
  • The change serializes RocksDB after Hadoop while preserving existing thirdparty build behavior.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant