| --- |
| # ---------------------------------------------------------------------------- |
| # |
| # *** AUTO GENERATED CODE *** Type: MMv1 *** |
| # |
| # ---------------------------------------------------------------------------- |
| # |
| # This file is automatically generated by Magic Modules and manual |
| # changes will be clobbered when the file is regenerated. |
| # |
| # Please read more about how to change this file in |
| # .github/CONTRIBUTING.md. |
| # |
| # ---------------------------------------------------------------------------- |
| subcategory: "Dataproc metastore" |
| description: |- |
| A managed metastore service that serves metadata queries. |
| --- |
| |
| # google\_dataproc\_metastore\_service |
| |
| A managed metastore service that serves metadata queries. |
| |
| |
| To get more information about Service, see: |
| |
| * [API documentation](https://cloud.google.com/dataproc-metastore/docs/reference/rest/v1/projects.locations.services) |
| * How-to Guides |
| * [Official Documentation](https://cloud.google.com/dataproc-metastore/docs/overview) |
| |
| <div class = "oics-button" style="float: right; margin: 0 0 -15px"> |
| <a href="https://console.cloud.google.com/cloudshell/open?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2Fterraform-google-modules%2Fdocs-examples.git&cloudshell_working_dir=dataproc_metastore_service_basic&cloudshell_image=gcr.io%2Fcloudshell-images%2Fcloudshell%3Alatest&open_in_editor=main.tf&cloudshell_print=.%2Fmotd&cloudshell_tutorial=.%2Ftutorial.md" target="_blank"> |
| <img alt="Open in Cloud Shell" src="//gstatic.com/cloudssh/images/open-btn.svg" style="max-height: 44px; margin: 32px auto; max-width: 100%;"> |
| </a> |
| </div> |
| ## Example Usage - Dataproc Metastore Service Basic |
| |
| |
| ```hcl |
| resource "google_dataproc_metastore_service" "default" { |
| service_id = "metastore-srv" |
| location = "us-central1" |
| port = 9080 |
| tier = "DEVELOPER" |
| |
| maintenance_window { |
| hour_of_day = 2 |
| day_of_week = "SUNDAY" |
| } |
| |
| hive_metastore_config { |
| version = "2.3.6" |
| } |
| |
| labels = { |
| env = "test" |
| } |
| } |
| ``` |
| ## Example Usage - Dataproc Metastore Service Cmek Example |
| |
| |
| ```hcl |
| resource "google_dataproc_metastore_service" "default" { |
| service_id = "example-service" |
| location = "us-central1" |
| |
| encryption_config { |
| kms_key = google_kms_crypto_key.crypto_key.id |
| } |
| |
| hive_metastore_config { |
| version = "3.1.2" |
| } |
| } |
| |
| resource "google_kms_crypto_key" "crypto_key" { |
| provider = google-beta |
| name = "example-key" |
| key_ring = google_kms_key_ring.key_ring.id |
| |
| purpose = "ENCRYPT_DECRYPT" |
| } |
| |
| resource "google_kms_key_ring" "key_ring" { |
| provider = google-beta |
| name = "example-keyring" |
| location = "us-central1" |
| } |
| ``` |
| ## Example Usage - Dataproc Metastore Service Private Service Connect |
| |
| |
| ```hcl |
| resource "google_compute_network" "net" { |
| name = "my-network" |
| auto_create_subnetworks = false |
| } |
| |
| resource "google_compute_subnetwork" "subnet" { |
| name = "my-subnetwork" |
| region = "us-central1" |
| network = google_compute_network.net.id |
| ip_cidr_range = "10.0.0.0/22" |
| private_ip_google_access = true |
| } |
| |
| resource "google_dataproc_metastore_service" "default" { |
| service_id = "metastore-srv" |
| location = "us-central1" |
| |
| hive_metastore_config { |
| version = "3.1.2" |
| } |
| |
| network_config { |
| consumers { |
| subnetwork = google_compute_subnetwork.subnet.id |
| } |
| } |
| } |
| ``` |
| ## Example Usage - Dataproc Metastore Service Private Service Connect Custom Routes |
| |
| |
| ```hcl |
| resource "google_compute_network" "net" { |
| provider = google-beta |
| name = "my-network" |
| auto_create_subnetworks = false |
| } |
| |
| resource "google_compute_subnetwork" "subnet" { |
| provider = google-beta |
| name = "my-subnetwork" |
| region = "us-central1" |
| network = google_compute_network.net.id |
| ip_cidr_range = "10.0.0.0/22" |
| private_ip_google_access = true |
| } |
| |
| resource "google_dataproc_metastore_service" "default" { |
| provider = google-beta |
| service_id = "metastore-srv" |
| location = "us-central1" |
| |
| hive_metastore_config { |
| version = "3.1.2" |
| } |
| |
| network_config { |
| consumers { |
| subnetwork = google_compute_subnetwork.subnet.id |
| } |
| custom_routes_enabled = true |
| } |
| } |
| ``` |
| <div class = "oics-button" style="float: right; margin: 0 0 -15px"> |
| <a href="https://console.cloud.google.com/cloudshell/open?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2Fterraform-google-modules%2Fdocs-examples.git&cloudshell_working_dir=dataproc_metastore_service_dpms2&cloudshell_image=gcr.io%2Fcloudshell-images%2Fcloudshell%3Alatest&open_in_editor=main.tf&cloudshell_print=.%2Fmotd&cloudshell_tutorial=.%2Ftutorial.md" target="_blank"> |
| <img alt="Open in Cloud Shell" src="//gstatic.com/cloudssh/images/open-btn.svg" style="max-height: 44px; margin: 32px auto; max-width: 100%;"> |
| </a> |
| </div> |
| ## Example Usage - Dataproc Metastore Service Dpms2 |
| |
| |
| ```hcl |
| resource "google_dataproc_metastore_service" "dpms2" { |
| service_id = "ms-dpms2" |
| location = "us-central1" |
| |
| # DPMS 2 requires SPANNER database type, and does not require |
| # a maintenance window. |
| database_type = "SPANNER" |
| |
| hive_metastore_config { |
| version = "3.1.2" |
| } |
| |
| scaling_config { |
| instance_size = "EXTRA_SMALL" |
| } |
| } |
| ``` |
| <div class = "oics-button" style="float: right; margin: 0 0 -15px"> |
| <a href="https://console.cloud.google.com/cloudshell/open?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2Fterraform-google-modules%2Fdocs-examples.git&cloudshell_working_dir=dataproc_metastore_service_dpms2_scaling_factor&cloudshell_image=gcr.io%2Fcloudshell-images%2Fcloudshell%3Alatest&open_in_editor=main.tf&cloudshell_print=.%2Fmotd&cloudshell_tutorial=.%2Ftutorial.md" target="_blank"> |
| <img alt="Open in Cloud Shell" src="//gstatic.com/cloudssh/images/open-btn.svg" style="max-height: 44px; margin: 32px auto; max-width: 100%;"> |
| </a> |
| </div> |
| ## Example Usage - Dataproc Metastore Service Dpms2 Scaling Factor |
| |
| |
| ```hcl |
| resource "google_dataproc_metastore_service" "dpms2_scaling_factor" { |
| service_id = "ms-dpms2sf" |
| location = "us-central1" |
| |
| # DPMS 2 requires SPANNER database type, and does not require |
| # a maintenance window. |
| database_type = "SPANNER" |
| |
| hive_metastore_config { |
| version = "3.1.2" |
| } |
| |
| scaling_config { |
| scaling_factor = "2" |
| } |
| } |
| ``` |
| |
| ## Argument Reference |
| |
| The following arguments are supported: |
| |
| |
| * `service_id` - |
| (Required) |
| The ID of the metastore service. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), |
| and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between |
| 3 and 63 characters. |
| |
| |
| - - - |
| |
| |
| * `labels` - |
| (Optional) |
| User-defined labels for the metastore service. |
| **Note**: This field is non-authoritative, and will only manage the labels present in your configuration. |
| Please refer to the field `effective_labels` for all of the labels present on the resource. |
| |
| * `network` - |
| (Optional) |
| The relative resource name of the VPC network on which the instance can be accessed. It is specified in the following form: |
| "projects/{projectNumber}/global/networks/{network_id}". |
| |
| * `port` - |
| (Optional) |
| The TCP port at which the metastore service is reached. Default: 9083. |
| |
| * `tier` - |
| (Optional) |
| The tier of the service. |
| Possible values are: `DEVELOPER`, `ENTERPRISE`. |
| |
| * `scaling_config` - |
| (Optional) |
| Represents the scaling configuration of a metastore service. |
| Structure is [documented below](#nested_scaling_config). |
| |
| * `maintenance_window` - |
| (Optional) |
| The one hour maintenance window of the metastore service. |
| This specifies when the service can be restarted for maintenance purposes in UTC time. |
| Maintenance window is not needed for services with the `SPANNER` database type. |
| Structure is [documented below](#nested_maintenance_window). |
| |
| * `encryption_config` - |
| (Optional) |
| Information used to configure the Dataproc Metastore service to encrypt |
| customer data at rest. |
| Structure is [documented below](#nested_encryption_config). |
| |
| * `hive_metastore_config` - |
| (Optional) |
| Configuration information specific to running Hive metastore software as the metastore service. |
| Structure is [documented below](#nested_hive_metastore_config). |
| |
| * `network_config` - |
| (Optional) |
| The configuration specifying the network settings for the Dataproc Metastore service. |
| Structure is [documented below](#nested_network_config). |
| |
| * `database_type` - |
| (Optional) |
| The database type that the Metastore service stores its data. |
| Default value is `MYSQL`. |
| Possible values are: `MYSQL`, `SPANNER`. |
| |
| * `release_channel` - |
| (Optional) |
| The release channel of the service. If unspecified, defaults to `STABLE`. |
| Default value is `STABLE`. |
| Possible values are: `CANARY`, `STABLE`. |
| |
| * `metadata_integration` - |
| (Optional) |
| The setting that defines how metastore metadata should be integrated with external services and systems. |
| Structure is [documented below](#nested_metadata_integration). |
| |
| * `telemetry_config` - |
| (Optional) |
| The configuration specifying telemetry settings for the Dataproc Metastore service. If unspecified defaults to JSON. |
| Structure is [documented below](#nested_telemetry_config). |
| |
| * `location` - |
| (Optional) |
| The location where the metastore service should reside. |
| The default value is `global`. |
| |
| * `project` - (Optional) The ID of the project in which the resource belongs. |
| If it is not provided, the provider project is used. |
| |
| |
| <a name="nested_scaling_config"></a>The `scaling_config` block supports: |
| |
| * `instance_size` - |
| (Optional) |
| Metastore instance sizes. |
| Possible values are: `EXTRA_SMALL`, `SMALL`, `MEDIUM`, `LARGE`, `EXTRA_LARGE`. |
| |
| * `scaling_factor` - |
| (Optional) |
| Scaling factor, in increments of 0.1 for values less than 1.0, and increments of 1.0 for values greater than 1.0. |
| |
| <a name="nested_maintenance_window"></a>The `maintenance_window` block supports: |
| |
| * `hour_of_day` - |
| (Required) |
| The hour of day (0-23) when the window starts. |
| |
| * `day_of_week` - |
| (Required) |
| The day of week, when the window starts. |
| Possible values are: `MONDAY`, `TUESDAY`, `WEDNESDAY`, `THURSDAY`, `FRIDAY`, `SATURDAY`, `SUNDAY`. |
| |
| <a name="nested_encryption_config"></a>The `encryption_config` block supports: |
| |
| * `kms_key` - |
| (Required) |
| The fully qualified customer provided Cloud KMS key name to use for customer data encryption. |
| Use the following format: `projects/([^/]+)/locations/([^/]+)/keyRings/([^/]+)/cryptoKeys/([^/]+)` |
| |
| <a name="nested_hive_metastore_config"></a>The `hive_metastore_config` block supports: |
| |
| * `endpoint_protocol` - |
| (Optional) |
| The protocol to use for the metastore service endpoint. If unspecified, defaults to `THRIFT`. |
| Default value is `THRIFT`. |
| Possible values are: `THRIFT`, `GRPC`. |
| |
| * `version` - |
| (Required) |
| The Hive metastore schema version. |
| |
| * `config_overrides` - |
| (Optional) |
| A mapping of Hive metastore configuration key-value pairs to apply to the Hive metastore (configured in hive-site.xml). |
| The mappings override system defaults (some keys cannot be overridden) |
| |
| * `kerberos_config` - |
| (Optional) |
| Information used to configure the Hive metastore service as a service principal in a Kerberos realm. |
| Structure is [documented below](#nested_kerberos_config). |
| |
| * `auxiliary_versions` - |
| (Optional) |
| A mapping of Hive metastore version to the auxiliary version configuration. |
| When specified, a secondary Hive metastore service is created along with the primary service. |
| All auxiliary versions must be less than the service's primary version. |
| The key is the auxiliary service name and it must match the regular expression a-z?. |
| This means that the first character must be a lowercase letter, and all the following characters must be hyphens, lowercase letters, or digits, except the last character, which cannot be a hyphen. |
| Structure is [documented below](#nested_auxiliary_versions). |
| |
| |
| <a name="nested_kerberos_config"></a>The `kerberos_config` block supports: |
| |
| * `keytab` - |
| (Required) |
| A Kerberos keytab file that can be used to authenticate a service principal with a Kerberos Key Distribution Center (KDC). |
| Structure is [documented below](#nested_keytab). |
| |
| * `principal` - |
| (Required) |
| A Kerberos principal that exists in the both the keytab the KDC to authenticate as. A typical principal is of the form "primary/instance@REALM", but there is no exact format. |
| |
| * `krb5_config_gcs_uri` - |
| (Required) |
| A Cloud Storage URI that specifies the path to a krb5.conf file. It is of the form gs://{bucket_name}/path/to/krb5.conf, although the file does not need to be named krb5.conf explicitly. |
| |
| |
| <a name="nested_keytab"></a>The `keytab` block supports: |
| |
| * `cloud_secret` - |
| (Required) |
| The relative resource name of a Secret Manager secret version, in the following form: |
| "projects/{projectNumber}/secrets/{secret_id}/versions/{version_id}". |
| |
| <a name="nested_auxiliary_versions"></a>The `auxiliary_versions` block supports: |
| |
| * `key` - (Required) The identifier for this object. Format specified above. |
| |
| * `version` - |
| (Required) |
| The Hive metastore version of the auxiliary service. It must be less than the primary Hive metastore service's version. |
| |
| * `config_overrides` - |
| (Optional) |
| A mapping of Hive metastore configuration key-value pairs to apply to the auxiliary Hive metastore (configured in hive-site.xml) in addition to the primary version's overrides. |
| If keys are present in both the auxiliary version's overrides and the primary version's overrides, the value from the auxiliary version's overrides takes precedence. |
| |
| <a name="nested_network_config"></a>The `network_config` block supports: |
| |
| * `consumers` - |
| (Required) |
| The consumer-side network configuration for the Dataproc Metastore instance. |
| Structure is [documented below](#nested_consumers). |
| |
| * `custom_routes_enabled` - |
| (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) |
| Enables custom routes to be imported and exported for the Dataproc Metastore service's peered VPC network. |
| |
| |
| <a name="nested_consumers"></a>The `consumers` block supports: |
| |
| * `endpoint_uri` - |
| (Output) |
| The URI of the endpoint used to access the metastore service. |
| |
| * `subnetwork` - |
| (Required) |
| The subnetwork of the customer project from which an IP address is reserved and used as the Dataproc Metastore service's endpoint. |
| It is accessible to hosts in the subnet and to all hosts in a subnet in the same region and same network. |
| There must be at least one IP address available in the subnet's primary range. The subnet is specified in the following form: |
| `projects/{projectNumber}/regions/{region_id}/subnetworks/{subnetwork_id} |
| |
| <a name="nested_metadata_integration"></a>The `metadata_integration` block supports: |
| |
| * `data_catalog_config` - |
| (Required) |
| The integration config for the Data Catalog service. |
| Structure is [documented below](#nested_data_catalog_config). |
| |
| |
| <a name="nested_data_catalog_config"></a>The `data_catalog_config` block supports: |
| |
| * `enabled` - |
| (Required) |
| Defines whether the metastore metadata should be synced to Data Catalog. The default value is to disable syncing metastore metadata to Data Catalog. |
| |
| <a name="nested_telemetry_config"></a>The `telemetry_config` block supports: |
| |
| * `log_format` - |
| (Optional) |
| The output format of the Dataproc Metastore service's logs. |
| Default value is `JSON`. |
| Possible values are: `LEGACY`, `JSON`. |
| |
| ## Attributes Reference |
| |
| In addition to the arguments listed above, the following computed attributes are exported: |
| |
| * `id` - an identifier for the resource with format `projects/{{project}}/locations/{{location}}/services/{{service_id}}` |
| |
| * `name` - |
| The relative resource name of the metastore service. |
| |
| * `endpoint_uri` - |
| The URI of the endpoint used to access the metastore service. |
| |
| * `state` - |
| The current state of the metastore service. |
| |
| * `state_message` - |
| Additional information about the current state of the metastore service, if available. |
| |
| * `artifact_gcs_uri` - |
| A Cloud Storage URI (starting with gs://) that specifies where artifacts related to the metastore service are stored. |
| |
| * `uid` - |
| The globally unique resource identifier of the metastore service. |
| |
| * `terraform_labels` - |
| The combination of labels configured directly on the resource |
| and default labels configured on the provider. |
| |
| * `effective_labels` - |
| All of labels (key/value pairs) present on the resource in GCP, including the labels configured through Terraform, other clients and services. |
| |
| |
| ## Timeouts |
| |
| This resource provides the following |
| [Timeouts](https://developer.hashicorp.com/terraform/plugin/sdkv2/resources/retries-and-customizable-timeouts) configuration options: |
| |
| - `create` - Default is 60 minutes. |
| - `update` - Default is 60 minutes. |
| - `delete` - Default is 60 minutes. |
| |
| ## Import |
| |
| |
| Service can be imported using any of these accepted formats: |
| |
| * `projects/{{project}}/locations/{{location}}/services/{{service_id}}` |
| * `{{project}}/{{location}}/{{service_id}}` |
| * `{{location}}/{{service_id}}` |
| |
| |
| In Terraform v1.5.0 and later, use an [`import` block](https://developer.hashicorp.com/terraform/language/import) to import Service using one of the formats above. For example: |
| |
| ```tf |
| import { |
| id = "projects/{{project}}/locations/{{location}}/services/{{service_id}}" |
| to = google_dataproc_metastore_service.default |
| } |
| ``` |
| |
| When using the [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import), Service can be imported using one of the formats above. For example: |
| |
| ``` |
| $ terraform import google_dataproc_metastore_service.default projects/{{project}}/locations/{{location}}/services/{{service_id}} |
| $ terraform import google_dataproc_metastore_service.default {{project}}/{{location}}/{{service_id}} |
| $ terraform import google_dataproc_metastore_service.default {{location}}/{{service_id}} |
| ``` |
| |
| ## User Project Overrides |
| |
| This resource supports [User Project Overrides](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#user_project_override). |