Information
It is recommended to create a sink that will export copies of all the log entries. This can help aggregate logs from multiple projects and export them to a Security Information and Event Management (SIEM).
Rationale:
Log entries are held in Cloud Logging. To aggregate logs, export them to a SIEM. To keep them longer, it is recommended to set up a log sink. Exporting involves writing a filter that selects the log entries to export, and choosing a destination in Cloud Storage, BigQuery, or Cloud Pub/Sub. The filter and destination are held in an object called a sink. To ensure all log entries are exported to sinks, ensure that there is no filter configured for a sink. Sinks can be created in projects, organizations, folders, and billing accounts.
Impact:
There are no costs or limitations in Cloud Logging for exporting logs, but the export destinations charge for storing or transmitting the log data.
Solution
From Google Cloud Console
Go to Logs Router by visiting https://console.cloud.google.com/logs/router.
Click on the arrow symbol with CREATE SINK text.
Fill out the fields for Sink details.
Choose Cloud Logging bucket in the Select sink destination drop down menu.
Choose a log bucket in the next drop down menu.
If an inclusion filter is not provided for this sink, all ingested logs will be routed to the destination provided above. This may result in higher than expected resource usage.
Click Create Sink.
For more information, see https://cloud.google.com/logging/docs/export/configure_export_v2#dest-create.
From Google Cloud CLI
To create a sink to export all log entries in a Google Cloud Storage bucket:
gcloud logging sinks create <sink-name> storage.googleapis.com/DESTINATION_BUCKET_NAME
Sinks can be created for a folder or organization, which will include all projects.
gcloud logging sinks create <sink-name> storage.googleapis.com/DESTINATION_BUCKET_NAME --include-children --folder=FOLDER_ID | --organization=ORGANIZATION_ID
Note:
A sink created by the command-line above will export logs in storage buckets. However, sinks can be configured to export logs into BigQuery, or Cloud Pub/Sub, or Custom Destination.
While creating a sink, the sink option --log-filter is not used to ensure the sink exports all log entries.
A sink can be created at a folder or organization level that collects the logs of all the projects underneath bypassing the option --include-children in the gcloud command.
Default Value:
By default, there are no sinks configured.