Repository navigation
Expand file tree
/
Copy path.readme-partials.yml
More file actions
239 lines (182 loc) · 9.75 KB
/
Copy path.readme-partials.yml
File metadata and controls
239 lines (182 loc) · 9.75 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
custom_content: |
## About Cloud Bigtable
[Cloud Bigtable][product-docs] is Google's NoSQL Big Data database service. It's
the same database that powers many core Google services, including Search, Analytics, Maps, and
Gmail.
Be sure to activate the Cloud Bigtable API and the Cloud Bigtable Admin API under APIs & Services in the GCP Console to use Cloud Bigtable from your project.
See the Bigtable client library documentation ([Admin API](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/package-summary.html) and [Data API](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/data/v2/package-summary.html)) to learn how to
interact with Cloud Bigtable using this Client Library.
## Concepts
Cloud Bigtable is composed of instances, clusters, nodes and tables.
### Instances
Instances are containers for clusters.
### Clusters
Clusters represent the actual Cloud Bigtable service. Each cluster belongs to a single Cloud Bigtable instance, and an instance can have up to 4 clusters. When your application
sends requests to a Cloud Bigtable instance, those requests are actually handled by one of the clusters in the instance.
### Nodes
Each cluster in a production instance has 3 or more nodes, which are compute resources that Cloud Bigtable uses to manage your data.
### Tables
Tables contain the actual data and are replicated across all of the clusters in an instance.
## Clients
The Cloud Bigtable API consists of:
### Data API
Allows callers to persist and query data in a table. It's exposed by [BigtableDataClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/data/v2/BigtableDataClient.html).
### Admin API
Allows callers to create and manage instances, clusters, tables, and access permissions. This API is exposed by: [BigtableInstanceAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableInstanceAdminClient.html) for Instance and Cluster level resources.
See [BigtableTableAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableTableAdminClient.html) for table management.
See [BigtableDataClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/data/v2/BigtableDataClient.html) for the data client.
See [BigtableInstanceAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableInstanceAdminClient.html) for the instance admin client.
See [BigtableTableAdminClient](https://googleapis.dev/java/google-cloud-clients/latest/com/google/cloud/bigtable/admin/v2/BigtableTableAdminClient.html) for the table admin client.
#### Calling Cloud Bigtable
The Cloud Bigtable API is split into 3 parts: Data API, Instance Admin API and Table Admin API.
Here is a code snippet showing simple usage of the Data API. Add the following imports
at the top of your file:
```java
import com.google.cloud.bigtable.data.v2.BigtableDataClient;
import com.google.cloud.bigtable.data.v2.models.Query;
import com.google.cloud.bigtable.data.v2.models.Row;
```
Then, to make a query to Bigtable, use the following code:
```java
// Instantiates a client
String projectId = "my-project";
String instanceId = "my-instance";
String tableId = "my-table";
// Create the client.
// Please note that creating the client is a very expensive operation
// and should only be done once and shared in an application.
BigtableDataClient dataClient = BigtableDataClient.create(projectId, instanceId);
try {
// Query a table
Query query = Query.create(tableId)
.range("a", "z")
.limit(26);
for (Row row : dataClient.readRows(query)) {
System.out.println(row.getKey());
}
} finally {
dataClient.close();
}
```
The Admin APIs are similar. Here is a code snippet showing how to create a table. Add the following
imports at the top of your file:
```java
import static com.google.cloud.bigtable.admin.v2.models.GCRules.GCRULES;
import com.google.cloud.bigtable.admin.v2.BigtableTableAdminClient;
import com.google.cloud.bigtable.admin.v2.models.CreateTableRequest;
import com.google.cloud.bigtable.admin.v2.models.Table;
```
Then, to create a table, use the following code:
```java
String projectId = "my-instance";
String instanceId = "my-database";
BigtableTableAdminClient tableAdminClient = BigtableTableAdminClient
.create(projectId, instanceId);
try {
tableAdminClient.createTable(
CreateTableRequest.of("my-table")
.addFamily("my-family")
);
} finally {
tableAdminClient.close();
}
```
TIP: If you are experiencing version conflicts with gRPC, see [Version Conflicts](#version-conflicts).
## Client side metrics
Cloud Bigtable client supports publishing client side metrics to
[Cloud Monitoring](https://cloud.google.com/monitoring/docs/monitoring-overview) under the
`bigtable.googleapis.com/client` namespace.
This feature is available once you upgrade to version 2.16.0 and above.
Follow the guide on https://cloud.google.com/bigtable/docs/client-side-metrics-setup to enable.
Since version 2.38.0, [client side metrics](https://cloud.google.com/bigtable/docs/client-side-metrics)
is enabled by default. This feature collects useful telemetry data in the client and is recommended to
use in conjunction with server-side metrics to get a complete, actionable view of your Bigtable
performance. There is no additional cost to publish and view client-side metrics
in Cloud Monitoring.
### Opt-out client side metrics
You can opt-out client side metrics with the following settings:
```java
BigtableDataSettings settings = BigtableDataSettings.newBuilder()
.setProjectId("my-project")
.setInstanceId("my-instance")
.setMetricsProvider(NoopMetricsProvider.INSTANCE)
.build();
```
### Use a custom OpenTelemetry instance
If your application already has OpenTelemetry integration, you can register client side metrics on
your OpenTelemetry instance. You can refer to
[CustomOpenTelemetryMetricsProvider](https://github.com/googleapis/java-bigtable/blob/main/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/stub/metrics/CustomOpenTelemetryMetricsProvider.java)
on how to set it up.
### Disable Bigtbale traces
If your application already has tracing integration and you want to disable Bigtable
traces, you can do the following:
```java
public static class MySampler extends Sampler {
private final Sampler childSampler;
MySampler(Sampler child) {
this.childSampler = child;
}
@Override
public boolean shouldSample(@Nullable SpanContext parentContext,
@Nullable Boolean hasRemoteParent,
TraceId traceId,
SpanId spanId,
String name,
List<Span> parentLinks) {
if (name.contains("Bigtable")) {
return false;
}
return childSampler.shouldSample(parentContext, hasRemoteParent, traceId, spanId, name, parentLinks);
}
@Override
public String getDescription() {
return "from my sampler";
}
}
```
And use this sampler in your trace config:
```java
Tracing.getTraceConfig().updateActiveTraceParams(
Tracing.getTraceConfig().getActiveTraceParams().toBuilder()
.setSampler(new MySampler(Samplers.probabilitySampler(0.1)))
.build()
);
```
## Version Conflicts
google-cloud-bigtable depends on gRPC directly which may conflict with the versions brought
in by other libraries, for example Apache Beam. This happens because internal dependencies
between gRPC libraries are pinned to an exact version of grpc-core
(see [here](https://github.com/grpc/grpc-java/commit/90db93b990305aa5a8428cf391b55498c7993b6e)).
If both google-cloud-bigtable and the other library bring in two gRPC libraries that depend
on the different versions of grpc-core, then dependency resolution will fail.
The easiest way to fix this is to depend on the gRPC bom, which will force all the gRPC
transitive libraries to use the same version.
Add the following to your project's pom.xml.
```
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-bom</artifactId>
<version>1.28.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
```
## Container Deployment
While deploying this client in [Google Kubernetes Engine(GKE)](https://cloud.google.com/kubernetes-engine) with [CoS](https://cloud.google.com/container-optimized-os/docs/). Please make sure to provide CPU configuration in your deployment file. With default configuration JVM detects only 1 CPU, which affects the number of channels with the client, resulting in performance repercussion.
Also, The number of `grpc-nio-worker-ELG-1-#` thread is same as number of CPUs. These are managed by a single `grpc-default-executor-#` thread, which is shared among multiple client instances.
For example:
```yaml
appVersion: v1
...
spec:
...
container:
resources:
requests:
cpu: "1" # Here 1 represents 100% of single node CPUs whereas other than 1 represents the number of CPU it would use from a node.
```
see [Assign CPU Resources to Containers](https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/#specify-a-cpu-request-and-a-cpu-limit) for more information.