Google Cloud Platform and API Call Attribution

When using multiple Projects, which Project an API call is attributed to is more complicated than one might think.

When using Google Cloud Platform (GCP), one often thinks about resources, e.g, storage buckets, instances, etc.; less apparent are the API calls made to access them. Even less apparent is that the frequency of these API calls is limited using Rate Quotas.

Rate quotas are typically used for limiting the number of requests you can make to an API or service. Rate quotas reset after a time interval that is specific to the service — for example, the number of API requests per day.

— Google Cloud — Working with Quotas

One might next think, how are these Rate Quotas enforced?

Quotas are enforced on a per-project basis, with the exception of the number of projects that you can create, which is enforced per user account and billing account.

— Google Cloud — Working with Quotas

When using multiple Projects, which Project an API call is attributed to is more complicated than one might think.

TL;DR:

• Some API calls, e.g., those supporting listing Google Cloud Engine (GCE) instances, are always attributed to the Project owning the resource
• Other API calls, e.g., those supporting writing log entries to Cloud Logging, are by default attributed to the Project owning the resource when a User is calling them; they are by default attributed to the Project owning the Service Account when a Service account is calling them
• Also for these other API calls, the API calls can be configured to be attributed to a billing (quota) Project. In this case, the User or Service Account must be granted the the Service Usage Consumer Role in the billing (quota) Project

Methodology

In each of scenarios, we made API calls using both the gcloud command-line tool (CLI) and the Client Libraries. In each case, we performed two sample operations:

• List Google Cloud Engine (GCE) instances
• Write a log entry to Cloud Logging

To allow the member (User or Service Account) to perform the operations, we granted it two Roles in each Project used.

• Viewer
• Logs Writer

The specific gcloud commands used are:

$ gcloud compute instances list
$ gcloud logging write debug 'hello world'

The Client Libraries are used in a sample Go application.

We then validated which Project an API call is attributed using Cloud Monitoring Metrics Explorer; using the following query:

Resource Type: consumed_api
Metric: serviceruntime.googleapis.com/api/request_count

The Project with data is the Project the API calls were attributed to.

gcloud CLI: User Logged into Project Accessing Resources in Same Project

In this simplest scenario, we started by authenticating the User and selecting the Project using the gcloud command-line tool:

$ gcloud init

Using the gcloud commands, we observed that the API calls were successful and attributed to the Project. At this point, it is not clear if the API calls are attributed to the Project the User is logged into or the Project owning the resource (because there is only one Project).

Client Libraries: User Logged in Accessing Resources in a Project

Authenticating the User to use the Client Libraries requires uses a separate gcloud command that does not involve selecting a Project:

$ gcloud auth application-default login

When we ran the command before running gcloud init, we observed the following warning; running the command after results in a different but similar warning.

WARNING: 
Cannot find a quota project to add to ADC. You might receive a "quota exceeded" or "API not enabled" error. Run $ gcloud auth application-default set-quota-project to add a quota project.

Ignoring the warning (will address later), we ran the sample Go application and observed that the API calls were successful and attributed to the Project. Here it is clear that the API calls are attributed to the Project owning the resource.

gcloud CLI: User Logged into Project Accessing Resources in a Different Project

As before, we authenticated the User and selected the Project. This time, however, we added the following option with a different Project to the commands:

--project PROJECT_ID

Using the gcloud commands, we observed that the API calls were successful and attributed to the Project owning the resource.

The conclusion we can draw from this and the two previous scenarios is that when a User is accessing resources, the Project owning the resource is attributed the API calls.

gcloud CLI: User Logged into Project Accessing Resources in a Different Project (Set Billing Project)

Building off the previous example where we added the project option, we next added the following option to control the billing (aka. quota) Project; the Project that API calls are attributed to:

--billing-project PROJECT_ID

A critical prerequisite in making this work is to grant the User the Service Usage Consumer Role in the billing Project.

Ability to inspect service states and operations, and consume quota and billing for a consumer project.

— Google Cloud — Understanding Roles

Using the gcloud commands, we then observed that the API calls were successful but their attribution varied between the two operations.

• List Google Cloud Engine (GCE) instances: Attributed to the Project owning the resource
• Write a log entry to Cloud Logging: Attributed to the billing (quota) Project

Before drawing conclusions, we repeated this scenario but with the Client Libraries.

Client Libraries: User Logged in Accessing Resources in a Project (Set Billing Project)

After authenticating to use the Client Libraries, we also set the billing (quota) Project using the following command; essentially what the earlier warning directed us to do:

gcloud auth application-default set-quota-project PROJECT_ID

We ran the sample Go application and observed that the API calls were successful but their attribution varied between the two operations (same as the previous scenario).

The conclusion we can draw from this and the previous scenario is that when a User is accessing resources, for some API calls the Project owning the resource is always attributed the API calls; for others, the API calls can be configured to be attributed to a billing (quota) Project.

gcloud CLI: Service Account in a Project Accessing Resources in Same Project

Having explored all the permutations of a User accessing resources, we switched to exploring scenarios involving a Service Account. In this scenario, we started by authenticating the Service Account using the gcloud command-line tool (requires creating and downloading a Service Account Key file).

gcloud auth activate-service-account SERVICE_ACCOUNT --key-file=KEY_FILE

As this command does not automatically select a Project, we added the following option with a same Project to the commands:

--project PROJECT_ID

Using the gcloud commands, we observed that the API calls were successful and attributed to the Project. As when using a User, it is not clear here if the API calls are attributed to the Project owning the Service Account or the Project owning the resource (because there is only one Project).

Client Libraries: Service Account in a Project Accessing Resources in Same Project

Authenticating the User to use the Client Libraries required setting an environment variable to the downloaded Service Account Key file.

export GOOGLE_APPLICATION_CREDENTIALS=KEY_FILE

We ran the sample Go application and observed that the API calls were successful and attributed to the Project. As in the previous scenario, it is not clear here if the API calls are attributed to the Project owning the Service Account or the Project owning the resource (because there is only one Project).

gcloud CLI: Service Account in a Project Accessing Resources in a Different Project

As before, we added the following option with a different Project to the commands:

--project PROJECT_ID

Using the gcloud commands, we then observed that the API calls were successful but their attribution varied between the two operations.

• List Google Cloud Engine (GCE) instances: Attributed to the Project owning the resource
• Write a log entry to Cloud Logging: Attributed to the Project owning the Service Account

Before drawing conclusions, we repeated this scenario but with the Client Libraries.

Client Libraries: Service Account in a Project Accessing Resources in a Different Project

We ran the sample Go application and observed that the API calls were successful but their attribution varied between the two operations (same as the previous scenario).

The conclusion we can draw from this and the previous scenario is that when a Service Account is accessing resources, for some API calls the Project owning the resource is attributed the API calls; for others, the API calls are attributed to the Project owning the Service Account.

gcloud CLI: Service Account in a Project Accessing Resources in a Different Project (Set Billing Project)

As in an earlier scenario, we next added the following option to control the billing (aka. quota) Project; the Project that API calls are attributed to:

--billing-project PROJECT_ID

Also, a critical prerequisite in making this work is to grant the Service Account the Service Usage Consumer Role in the billing Project.

Using the gcloud commands, we then observed that the API calls were successful but their attribution varied between the two operations.

• List Google Cloud Engine (GCE) instances: Attributed to the Project owning the resource
• Write a log entry to Cloud Logging: Attributed to the billing (quota) Project

Before drawing conclusions, we repeated this scenario but with the Client Libraries.

Client Libraries: Service Account in a Project Accessing Resources in a Different Project (Set Billing Project)

Previously, we used a gcloud command to authenticate the User to the Client Libraries and used another gcloud command to set the billing (quota) Project. In the case of a Service Account, we instead set the GOOGLE_APPLICATION_CREDENTIALS environment variable to authenticate it; as such we needed another mechanism to set the billing (quota) Project.

In this scenario, we updated (uncommented the code) the Go application to set the billing (quota) Project; the key function is WithQuotaProject:

func WithQuotaProject(quotaProject string) ClientOption
WithQuotaProject returns a ClientOption that specifies the project used for quota and billing purposes.

— Google Cloud — option

We ran the sample Go application and observed that the API calls were successful but their attribution varied between the two operations (same as the previous scenario).

The conclusion we can draw from this and the previous scenario is that when a Service Account is accessing resources, for some API calls the Project owning the resource is always attributed the API calls; for others, the API calls can be configured to be attributed to a billing (quota) Project.

Conclusion

Weaving all these conclusions together, we end up with:

• Some API calls, e.g., those supporting listing Google Cloud Engine (GCE) instances, are always attributed to the Project owning the resource
• Other API calls, e.g., those supporting writing log entries to Cloud Logging, are by default attributed to the Project owning the resource when a User is calling them; they are by default attributed to the Project owning the Service Account when a Service account is calling them
• Also for these other API calls, the API calls can be configured to be attributed to a billing (quota) Project. In this case, the User or Service Account must be granted the the Service Usage Consumer Role in the billing (quota) Project