Visitar URL original
Ruby: build extensions for arm64-darwin by qnighy · Pull Request #8232 · protocolbuffers/protobuf · GitHub
Skip to content

Ruby: build extensions for arm64-darwin - #8232

Merged
deannagarcia merged 3 commits into
protocolbuffers:masterfrom
qnighy:arm64-darwin
Oct 14, 2021
Merged

Ruby: build extensions for arm64-darwin#8232
deannagarcia merged 3 commits into
protocolbuffers:masterfrom
qnighy:arm64-darwin

Conversation

@qnighy

@qnighy qnighy commented Jan 25, 2021

Copy link
Copy Markdown
Contributor

This PR adds builds for arm64-darwin. It also renames universal-darwin back to x86_64-darwin to avoid confusion. These arch names are already used in nokogiri, so I think it's fine to rename it back to them.

See #8231 for a different approach.

@google-cla google-cla Bot added the cla: yes label Jan 25, 2021
@qnighy qnighy mentioned this pull request Jan 25, 2021
Closed
@haberman

haberman commented Jan 27, 2021

Copy link
Copy Markdown
Member

I'm seeing the following error in the "MacOS Ruby Release" test. Perhaps this means we need our macOS build machine to be updated to something that supports aarch64?

Found Ruby version 3.0.0 for platform aarch64-darwin (/Users/kbuilder/.rake-compiler/ruby/arm64-darwin/ruby-3.0.0/lib/ruby/3.0.0/aarch64-darwin/rbconfig.rb)
Found Ruby version 2.7.0 for platform x86_64-darwin (/Users/kbuilder/.rake-compiler/ruby/x86_64-darwin/ruby-2.7.0/lib/ruby/2.7.0/x86_64-darwin/rbconfig.rb)
Found Ruby version 3.0.0 for platform x86_64-darwin (/Users/kbuilder/.rake-compiler/ruby/x86_64-darwin/ruby-3.0.0/lib/ruby/3.0.0/x86_64-darwin/rbconfig.rb)
+ rake -f tmpfile.KytCYhXz cross-ruby VERSION=2.7.0 HOST=arm64-darwin 'MAKE=make -j8'
mkdir -p /Users/kbuilder/.rake-compiler/builds/arm64-darwin/ruby-2.7.0
cd /Users/kbuilder/.rake-compiler/builds/arm64-darwin/ruby-2.7.0
/Users/kbuilder/.rake-compiler/sources/ruby-2.7.0/configure --host=arm64-darwin --target=arm64-darwin --build=x86_64-apple-darwin18.7.0 --enable-static --disable-shared --disable-install-doc --without-gmp --with-ext= LDFLAGS=-pipe --prefix=/Users/kbuilder/.rake-compiler/ruby/arm64-darwin/ruby-2.7.0
checking for ruby... /Users/kbuilder/.rvm/rubies/ruby-2.7.0/bin/ruby
tool/config.guess already exists
tool/config.sub already exists
checking build system type... x86_64-apple-darwin18.7.0
checking host system type... Invalid configuration `arm64-darwin': machine `arm64-unknown' not recognized
configure: error: /bin/sh /Users/kbuilder/.rake-compiler/sources/ruby-2.7.0/tool/config.sub arm64-darwin failed
rake aborted!
Command failed with status (1): [/Users/kbuilder/.rake-compiler/sources/rub...]

@dlj-NaN

dlj-NaN commented Feb 3, 2021

Copy link
Copy Markdown
Contributor

What is the reason for splitting apart the darwins, instead of just using darwin-universal?

@qnighy

qnighy commented Feb 3, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

What is the reason for splitting apart the darwins, instead of just using darwin-universal?

Because there's no such architecture to compile Ruby for. The following command fails:

$rake -f tasks/bin/cross-ruby.rake cross-ruby VERSION="3.0.0" HOST=universal-darwin
mkdir -p /Users/qnighy/.rake-compiler/builds/universal-darwin/ruby-3.0.0
cd /Users/qnighy/.rake-compiler/builds/universal-darwin/ruby-3.0.0
/Users/qnighy/.rake-compiler/sources/ruby-3.0.0/configure --host=universal-darwin --target=universal-darwin --build=x86_64-apple-darwin19.4.0 --enable-static --disable-shared --disable-install-doc --without-gmp --with-ext= LDFLAGS=-pipe --prefix=/Users/qnighy/.rake-compiler/ruby/universal-darwin/ruby-3.0.0
checking for ruby... /Users/qnighy/.rbenv/versions/2.7.1/bin/ruby
tool/config.guess already exists
tool/config.sub already exists
checking build system type... x86_64-apple-darwin19.4.0
checking host system type... Invalid configuration `universal-darwin': machine `universal-unknown' not recognized
configure: error: /bin/sh /Users/qnighy/.rake-compiler/sources/ruby-3.0.0/tool/config.sub universal-darwin failed
rake aborted!
Command failed with status (1): [/Users/qnighy/.rake-compiler/sources/ruby-...]
/Users/qnighy/workdir/rake-compiler/tasks/bin/cross-ruby.rake:143:in `block (2 levels) in <top (required)>'
/Users/qnighy/workdir/rake-compiler/tasks/bin/cross-ruby.rake:140:in `block in <top (required)>'
Tasks: TOP => cross-ruby => install => /Users/qnighy/.rake-compiler/ruby/universal-darwin/ruby-3.0.0/bin/ruby.exe => /Users/qnighy/.rake-compiler/builds/universal-darwin/ruby-3.0.0/ruby.exe => /Users/qnighy/.rake-compiler/builds/universal-darwin/ruby-3.0.0/Makefile
(See full trace by running task with --trace)

Previously we just used x86_64-darwin as a fake universal-darwin, which is not universal at all. That's what the following lines did:

sed 's/x86_64-darwin-11/universal-darwin/' ~/.rake-compiler/config.yml > "$CROSS_RUBY"
mv "$CROSS_RUBY" ~/.rake-compiler/config.yml

@qnighy

qnighy commented Feb 3, 2021

Copy link
Copy Markdown
Contributor Author

I'm seeing the following error in the "MacOS Ruby Release" test. Perhaps this means we need our macOS build machine to be updated to something that supports aarch64?

That's probably old autoconf's problem. Ruby 3.0 builds on my local machine but Ruby 2.7 not. I should have checked that.

@qnighy

qnighy commented Feb 3, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Moreover, in my local machine, it seems to produce x86_64 binaries regardless of HOST. I needed to specify the following environments to actually produce aarch64 binaries:

CC=clang --target=arm64-darwin
CXX=clang++ --target=arm64-darwin
CPP=clang -E --target=arm64-darwin
CPPFLAGS=-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
CFLAGS=-Wno-nullability-completeness
LDFLAGS=-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/system -F/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/

I'll incorporate it into the PR later.

@bouk

bouk commented Mar 23, 2021 •
edited
Loading

Copy link
Copy Markdown

Any progress on this? I guess we need a googler to do something with this.

@bouk

bouk commented May 21, 2021 •
edited
Loading

Copy link
Copy Markdown

Maybe you could update this PR to just build x86_64 for Darwin, and rely on the installing machine to build for arm64. I.e. just remove the universal-darwin name

@qnighy

qnighy commented Sep 29, 2021

Copy link
Copy Markdown
Contributor Author

That's probably old autoconf's problem.

This part was actually easy, I just needed to use aarch64-darwin instead of arm64-darwin.

@deannagarcia
deannagarcia self-requested a review October 14, 2021 18:04
@deannagarcia

deannagarcia commented Oct 14, 2021

Copy link
Copy Markdown
Contributor

Sorry for the delay and thanks for the contribution. Everything's passing right now, so going to merge this in!

@deannagarcia
deannagarcia merged commit 3d223dc into protocolbuffers:master Oct 14, 2021
@mloughran mloughran mentioned this pull request Oct 25, 2021
Closed
@ngan

ngan commented Oct 25, 2021

Copy link
Copy Markdown

@qnighy was this PR suppose produce a arm64-darwin variant? If so, unfortunately, I'm not seeing it on Rubygems:
image

@jhahm981 jhahm981 mentioned this pull request Feb 7, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@deannagarcia deannagarcia deannagarcia approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla: yes ruby

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@qnighy @haberman @dlj-NaN @bouk @deannagarcia @ngan @acozzette @protobuf-kokoro