Visitar URL original
Fix Kokoro release jobs (#4225) · googleapis/google-cloud-java@fb2b690 · GitHub
Skip to content

Commit fb2b690

Browse files
chingor13chingor13
authored
Fix Kokoro release jobs (#4225)
* Fix Kokoro release jobs. Needs loopback pinentry-mode * Skip install on deploy * skip stagingProfileId * Capture built artifacts * Enable promote/drop jobs via environment variable * Fix directory path and use -B for drop/release * promote/drop need the settings file * Fix path to settings file * Update RELEASING documentation to use the Kokoro jobs
1 parent e9ea3d9 commit fb2b690

9 files changed

Lines changed: 152 additions & 68 deletions

File tree

‎.kokoro/release/drop.sh‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,20 @@
1515

1616
set -eo pipefail
1717

18+
# STAGING_REPOSITORY_ID must be set
19+
if [ -z "${STAGING_REPOSITORY_ID}" ]; then
20+
echo "Missing STAGING_REPOSITORY_ID environment variable"
21+
exit 1
22+
fi
23+
1824
source $(dirname "$0")/common.sh
19-
MAVEN_SETTINGS_FILE=$(realpath $(dirname "$0")/../)/settings.xml
20-
pushd $(dirname "$0")/../
25+
MAVEN_SETTINGS_FILE=$(realpath $(dirname "$0")/../../)/settings.xml
26+
pushd $(dirname "$0")/../../
2127

2228
setup_environment_secrets
2329
create_settings_xml_file "settings.xml"
2430

25-
mvn nexus-staging:drop -P release
31+
mvn nexus-staging:drop -B \
32+
-P release \
33+
--settings ${MAVEN_SETTINGS_FILE} \
34+
-DstagingRepositoryId=${STAGING_REPOSITORY_ID}

‎.kokoro/release/promote.sh‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,20 @@
1515

1616
set -eo pipefail
1717

18+
# STAGING_REPOSITORY_ID must be set
19+
if [ -z "${STAGING_REPOSITORY_ID}" ]; then
20+
echo "Missing STAGING_REPOSITORY_ID environment variable"
21+
exit 1
22+
fi
23+
1824
source $(dirname "$0")/common.sh
19-
MAVEN_SETTINGS_FILE=$(realpath $(dirname "$0")/../)/settings.xml
20-
pushd $(dirname "$0")/../
25+
MAVEN_SETTINGS_FILE=$(realpath $(dirname "$0")/../../)/settings.xml
26+
pushd $(dirname "$0")/../../
2127

2228
setup_environment_secrets
2329
create_settings_xml_file "settings.xml"
2430

25-
mvn nexus-staging:release -P release
31+
mvn nexus-staging:release -B \
32+
-P release \
33+
--settings ${MAVEN_SETTINGS_FILE} \
34+
-DstagingRepositoryId=${STAGING_REPOSITORY_ID}

‎.kokoro/release/stage.cfg‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,11 @@
66
# Need to save the properties file
77
action {
88
define_artifacts {
9-
regex: "github/google-cloud-java/target/nexus-staging/staging/*.properties"
10-
strip_prefix: "github/google-cloud-java"
9+
regex: "**/target/nexus-staging/staging/*.properties"
10+
regex: "**/target/*.jar"
11+
regex: "**/target/*.jar.asc"
12+
regex: "**/target/*.pom"
13+
regex: "**/target/*.pom.asc"
14+
fail_if_no_artifacts: true
1115
}
1216
}

‎.kokoro/release/stage.sh‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,9 @@ pushd $(dirname "$0")/../../
2222
setup_environment_secrets
2323
create_settings_xml_file "settings.xml"
2424

25-
mvn clean install deploy \
25+
mvn clean deploy -B \
2626
-DskipTests=true \
2727
--settings ${MAVEN_SETTINGS_FILE} \
28-
-DstagingProfileId=3187e4f20d328b \
2928
-Dgpg.executable=gpg \
3029
-Dgpg.passphrase=${GPG_PASSPHRASE} \
3130
-Dgpg.homedir=${GPG_HOMEDIR} \

‎RELEASING.md‎

Lines changed: 97 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,97 @@
1-
One-time setup
2-
==============
1+
## Setup
2+
3+
### Install releasetool
4+
5+
See [releasetool](https://github.com/googleapis/releasetool) for installation instructions. You will
6+
need python 3.6+ to run this tool.
7+
8+
## Prepare a release
9+
10+
1. Make sure the team agrees that it is time to release.
11+
12+
2. Look over all of the commits since the last release and make sure there are no breaking changes
13+
on the public surface. If there are any breaking changes, create and merge a new PR to revert the
14+
surface back.
15+
16+
**Note:** this should just be a scan of the public surface that would appear in Java doc.
17+
Implementation changes, README changes, and snippet changes can all be skipped for this check.
18+
19+
3. Verify that all unit and integration tests for the last commit have passed.
20+
21+
4. Run `releasetool start`. Select "minor" or "patch" for the release type. This will bump the
22+
artifact versions, ask you to edit release notes, and create the release pull request.
23+
24+
**Note:** be sure to make these notes nice as they will be used for the release notes as well.
25+
26+
## Pushing a release to Maven using Kokoro
27+
28+
To manually publish the artifacts to Maven (rather than using Kokoro), you will need to follow the
29+
"Additional setup for manual publishing" steps below and follow the instructions in the
30+
"Manually publishing" section.
31+
32+
1. Trigger the `google-cloud-java/release/stage` Kokoro job and wait for it to complete. This will
33+
stage the built artifacts and prepare them for publishing.
34+
35+
2. Look through the logs for the `google-cloud-java/release/stage` and find the staging repository
36+
ids used. They will look like `comgoogleapi-XYZ` and `comgooglecloud-XYZ`.
37+
38+
3. Promote or drop the staged repository.
39+
40+
a. To publish the staged repository, trigger the `google-cloud-java/release/promote` Kokoro job for
41+
each staging repository. To specify the staging repository, add an environment variable
42+
configuration with `STAGING_REPOSITORY_ID=<staging repository id>` from the UI. **Note: thi
43+
will need to be run for each staging repository. It may take a few hours for the released
44+
versions to be available for users to download.**
45+
46+
b. To drop (abort) the staged repository, trigger the `google-cloud-java/release/drop` Kokoro job
47+
with the same staging repository id configuration as if you were publishing.
48+
49+
## Publish Javadoc
50+
51+
1. Run `git clean -x -f -d` to put the repo in a clean state.
52+
53+
2. Locally build the repo by running `mvn install -DskipTests`.
54+
55+
3. Run `python utilities/stage_sites.py`. This script checks out `gh-pages` branch of the
56+
repository, builds the documentation site and javadocs, copies them to the branch and commits it.
57+
This script does not push the docs and it must be done manually on the later step. The script
58+
assumes that there is no directory called `tmp_gh-pages` in the repository root. If it is
59+
present, remove it before running the script.
60+
61+
4. Run `cd tmp_gh-pages && git push && cd ..`.
62+
63+
5. (Optional) Run `rm -rf tmp_gh-pages` to remove the generated docs directory from your local
64+
machine.
65+
66+
## Tag the release
67+
68+
1. Run `releasetool tag` to publish a release on Github. It will list the last few merged PRs.
69+
Select the newly merged release PR. Releasetool will create the GitHub release with notes
70+
extracted from the pull request and tag the new release.
71+
72+
## Prepare the next snapshot version
73+
74+
1. Run `releasetool start` to bump the next snapshot version. Select "snapshot" when prompted for
75+
the release type. This will bump the artifact versions and create a pull request.
76+
77+
2. Review and submit the PR.
78+
79+
## Additional setup for manual publishing
80+
81+
### Set up Sonatype Account
382

4-
Set up Sonatype Account
5-
-----------------------
683
* Sign up for a Sonatype JIRA account [here](https://issues.sonatype.org)
784
* Click *Sign Up* in the login box, follow instructions
885

9-
Get access to repository
10-
------------------------
86+
### Get access to repository
87+
1188
* Go to [community support](https://issues.sonatype.org/browse/OSSRH)
1289
* Ask for publish rights by creating an issue similar to [this one](https://issues.sonatype.org/browse/OSSRH-32032)
1390
* You must be logged in to create a new issue
1491
* Use the *Create* button at the top tab
1592

16-
Set up PGP keys
17-
---------------
93+
### Set up PGP keys
94+
1895
* Install GNU Privacy Guard (GPG)
1996
* GPG is installed by default on Ubuntu systems
2097
* For other systems, see [GnuPG download page](https://www.gnupg.org/download/)
@@ -30,8 +107,8 @@ Set up PGP keys
30107

31108
* Upload your public key to a public server: ```gpg --send-keys --keyserver hkp://pgp.mit.edu ABCDEFGH```
32109

33-
Create a Maven settings file
34-
----------------------------
110+
### Create a Maven settings file
111+
35112
* Create a file at ```$HOME/.m2/settings.xml``` with your passphrase and your sonatype username and password
36113
```
37114
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
@@ -72,42 +149,13 @@ Create a Maven settings file
72149

73150
Make sure you are using Maven version 3.3 or higher to support the Nexus plugin required to stage a release.
74151

75-
Install releasetool
76-
-------------------
77-
See [releasetool](https://github.com/googleapis/releasetool) for installation instructions. You will
78-
need python 3.6+ to run this tool.
79-
80-
To push a release version
81-
=========================
152+
### Manually publishing (from a local machine)
82153

83-
1. Make sure the team agrees that it is time to release.
154+
1. Run `git clean -x -f -d` to put the repo in a clean state.
84155

85-
2. Look over all of the commits since the last release and make sure there are no breaking changes
86-
on the public surface. If there are any breaking changes, create and merge a new PR to revert the
87-
surface back.
88-
89-
**Note:** this should just be a scan of the public surface that would appear in Java doc.
90-
Implementation changes, README changes, and snippet changes can all be skipped for this check.
156+
2. Locally build the repo by running `mvn install -DskipTests`.
91157

92-
3. Verify that all unit and integration tests for the last commit have passed.
93-
94-
4. Run `releasetool start`. Select "minor" or "patch" for the release type. This will bump the
95-
artifact versions, ask you to edit release notes, and create the release pull request.
96-
97-
**Note:** be sure to make these notes nice as they will be used for the release notes as well.
98-
99-
5. Run `git clean -x -f -d` to put the repo in a clean state.
100-
101-
6. Locally build the repo by running `mvn install -DskipTests`.
102-
103-
7. Run `python utilities/stage_sites.py`. This script checks out `gh-pages` branch of the
104-
repository, builds the documentation site and javadocs, copies them to the branch and commits it.
105-
This script does not push the docs and it must be done manually on the later step. The script
106-
assumes that there is no directory called `tmp_gh-pages` in the repository root. If it is
107-
present, remove it before running the script.
108-
109-
8. Check that you are not trying to release a SNAPSHOT build (the artifacts versions do not have
110-
"-SNAPSHOT" suffix) and then run `mvn clean deploy -DskipTests=true --settings ~/.m2/settings.xml -P release`
158+
3. and then run `mvn clean deploy -DskipTests=true --settings ~/.m2/settings.xml -P release`
111159
command. It will build and deploy artifacts to the staging repository.
112160

113161
**Note:** you may need to specify the stagingProfileId with `-DstagingProfileId=3187e4f20d328b`
@@ -116,20 +164,11 @@ To push a release version
116164
9. Run `mvn nexus-staging:release` to release the artifacts. If you wish to abort the release, run
117165
`mvn nexus-staging:drop`.
118166

119-
10. Run `cd tmp_gh-pages && git push && cd ..` to push the previously generated docs (step 7).
120-
121-
11. Run `rm -rf tmp_gh-pages` to remove the generated docs directory from your local machine.
122-
123-
12. Run `releasetool tag` to publish a release on Github. It will list the last few merged PRs.
124-
Select the newly merged release PR. Releasetool will create the GitHub release with notes
125-
extracted from the pull request and tag the new release.
126-
127-
13. Run `releasetool start` to bump the next snapshot version. Select "snapshot" when prompted for
128-
the release type. This will bump the artifact versions and create a pull request.
129-
130-
14. Review and submit the PR.
167+
## Improvements
131168

132-
Improvements
133-
============
169+
* Automatic tagging is not currently implemented, though it was discussed in #119. If the version
170+
updates continue to be manual, a one-line git tag command can be added to `after_success.sh` to
171+
correctly tag releases. However, automatically creating useful annotations for this tag will be
172+
difficult. Also, if the release process becomes fully automated, tagging becomes a harder
173+
problem, as mentioned in that issue.
134174

135-
Automatic tagging is not currently implemented, though it was discussed in [#119](https://github.com/googleapis/google-cloud-java/pull/119). If the version updates continue to be manual, a one-line git tag command can be added to `after_success.sh` to correctly tag releases. However, automatically creating useful annotations for this tag will be difficult. Also, if the release process becomes fully automated, tagging becomes a harder problem, as mentioned in that issue.

‎google-api-grpc/pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -960,6 +960,12 @@
960960
<goals>
961961
<goal>sign</goal>
962962
</goals>
963+
<configuration>
964+
<gpgArguments>
965+
<arg>--pinentry-mode</arg>
966+
<arg>loopback</arg>
967+
</gpgArguments>
968+
</configuration>
963969
</execution>
964970
</executions>
965971
</plugin>

‎google-cloud-bom/pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1034,6 +1034,12 @@
10341034
<goals>
10351035
<goal>sign</goal>
10361036
</goals>
1037+
<configuration>
1038+
<gpgArguments>
1039+
<arg>--pinentry-mode</arg>
1040+
<arg>loopback</arg>
1041+
</gpgArguments>
1042+
</configuration>
10371043
</execution>
10381044
</executions>
10391045
</plugin>

‎google-cloud-clients/pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -942,6 +942,12 @@
942942
<goals>
943943
<goal>sign</goal>
944944
</goals>
945+
<configuration>
946+
<gpgArguments>
947+
<arg>--pinentry-mode</arg>
948+
<arg>loopback</arg>
949+
</gpgArguments>
950+
</configuration>
945951
</execution>
946952
</executions>
947953
</plugin>

‎google-cloud-testing/google-cloud-bigtable-emulator/pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -225,6 +225,12 @@
225225
<goals>
226226
<goal>sign</goal>
227227
</goals>
228+
<configuration>
229+
<gpgArguments>
230+
<arg>--pinentry-mode</arg>
231+
<arg>loopback</arg>
232+
</gpgArguments>
233+
</configuration>
228234
</execution>
229235
</executions>
230236
</plugin>

0 commit comments

Comments
 (0)