v1.14.5/website/content/docs/upgrading/upgrade-to-1.12.x.mdx - hashicorp/vault - Git at Google

 ---
 layout: docs
 page_title: Upgrading to Vault 1.12.x - Guides
 description: |-
   This page contains the list of deprecations and important or breaking changes
   for Vault 1.12.x. Please read it carefully.
 ---

 # Overview

 This page contains the list of deprecations and important or breaking changes
 for Vault 1.12.x compared to 1.11. Please read it carefully.

 ## Changes

 ### Supported storage backends

 Vault Enterprise will now perform a supported storage check at startup. There is no impact on Vault Community Edition users.

 @include 'ent-supported-storage.mdx'

 @include 'consul-dataplane-upgrade-note.mdx'

 ### External plugin loading

 Vault 1.12.0 introduced a change to how external plugins are loaded. Prior to
 Vault 1.12.0 plugins were lazy loaded on startup. This means that plugin
 processes were killed after a successful mount and then respawned when a
 request is routed to them. Vault 1.12.0 introduced auto mutual TLS for
 secrets/auth plugins so we do not lazy load them on startup anymore.

 ## Known issues

 ### Pinning to builtin plugin versions may cause failure on upgrade

 1.12.0 introduced plugin versions, and with it, the ability to explicitly specify
 the builtin version of a plugin when mounting an auth, database or secrets plugin.
 For example, `vault auth enable -plugin-version=v1.12.0+builtin.vault approle`. If
 there are any mounts where the _builtin_ version was explicitly specified in this way,
 Vault may fail to start on upgrading to 1.12.1 due to the specified version no
 longer being available.

 To check whether a mount path is affected, read the tune information, or the
 database config. The affected plugins are `snowflake-database-plugin@v0.6.0+builtin`
 and any plugins with `+builtin.vault` metadata in their version.

 In this example, the first two mounts are affected because `plugin_version` is
 explicitly set and is one of the affected versions. The third mount is not
 affected because it only has `+builtin` metadata, and is not the Snowflake
 database plugin. All mounts where the version is omitted, or the plugin is
 external (regardless of whether the version is specified) are unaffected.

 -> **NOTE:** Make sure you use Vault CLI 1.12.0 or later to check mounts.

 ```shell-session
 $ vault read sys/auth/approle/tune
 Key                  Value
 ---                  -----
 ...
 plugin_version       v1.12.0+builtin.vault

 $ vault read database/config/snowflake
 Key                                   Value
 ---                                   -----
 ...
 plugin_name                           snowflake-database-plugin
 plugin_version                        v0.6.0+builtin

 $ vault read sys/auth/kubernetes/tune
 Key                  Value
 ---                  -----
 ...
 plugin_version       v0.14.0+builtin
 ```

 As it is not currently possible to unset the plugin version, there are 3 possible
 remediations if you have any affected mounts:

 * Upgrade Vault directly to 1.12.2 once released
 * Upgrade to an external version of the plugin before upgrading to 1.12.1;
   * Using the [tune API](/vault/api-docs/system/auth#tune-auth-method) for auth methods
   * Using the [tune API](/vault/api-docs/system/mounts#tune-mount-configuration) for secrets plugins
   * Or using the [configure connection](/vault/api-docs/secret/databases#configure-connection)
     API for database plugins
 * Unmount and remount the path without a version specified before upgrading to 1.12.1.
   **Note:** This will delete all data and leases associated with the mount.

 The bug was introduced by commit
 https://github.com/hashicorp/vault/commit/c36330f4c713b886a8a23c08cbbd862a7c530fc8.

 #### Impacted versions

 Affects upgrading from 1.12.0 to 1.12.1. All other upgrade paths are unaffected.
 1.12.2 will introduce a fix that enables upgrades from affected deployments of
 1.12.0.

 ### Mounts associated with deprecated builtin plugins will result in core shutdown on upgrade

 As of 1.12.0 Standalone (logical) DB Engines and the AppId Auth Method have been
 marked with the `Pending Removal` status. Any attempt to unseal Vault with
 mounts backed by one of these builtin plugins will result in an immediate
 shutdown of the Vault core.

 -> **NOTE** In the event that an external plugin with the same name and type as
 a deprecated builtin is deregistered, any subsequent unseal of Vault will also
 result in a core shutdown.

 ```shell-session
 $ vault plugin register -sha256=c805cf3b69f704dfcd5176ef1c7599f88adbfd7374e9c76da7f24a32a97abfe1 auth app-id
 Success! Registered plugin: app-id
 $ vault auth enable -plugin-name=app-id plugin
 Success! Enabled app-id auth method at: app-id/
 $ vault auth list -detailed
 app-id/    app-id    auth_app-id_3a8f2e24    system         system     default-service    replicated     false        false                      map[]      n/a                        0018263c-0d64-7a70-fd5c-50e05c5f5dc3    n/a        n/a                      c805cf3b69f704dfcd5176ef1c7599f88adbfd7374e9c76da7f24a32a97abfe1    n/a
 $ vault plugin deregister auth app-id
 Success! Deregistered plugin (if it was registered): app-id
 $ vault plugin list -detailed | grep "app-id"
 app-id                               auth        v1.12.0+builtin.vault                                 pending removal
 ```

 The remediation for affected mounts is to set the
 [VAULT_ALLOW_PENDING_REMOVAL_MOUNTS](/vault/docs/commands/server#vault_allow_pending_removal_mounts)
 environment variable and replace any `Pending Removal` feature with the
 [preferred alternative
 feature](/vault/docs/deprecation/faq#q-what-should-i-do-if-i-use-mount-filters-appid-or-any-of-the-standalone-db-engines).

 For more information on the phases of deprecation, see the [Deprecation Notices
 FAQ](/vault/docs/deprecation/faq#q-what-are-the-phases-of-deprecation).

 #### Impacted versions

 Affects upgrading from any version of Vault to 1.12.x. All other upgrade paths
 are unaffected.

 ### `vault plugin list` fails when audit logging is enabled

 If audit logging is enabled, Vault will fail to audit the response from any
 calls to the [`GET /v1/sys/plugins/catalog`](/vault/api-docs/system/plugins-catalog#list-plugins)
 endpoint, which causes the whole request to fail and return a 500 internal
 server error. From the CLI, this looks like the following:

 ```shell-session
 $ vault plugin list
 Error listing available plugins: data from server response is empty
 ```

 It will produce errors in Vault Server's logs such as:

 ```text
 2022-11-30T20:04:22.397Z [ERROR] audit: panic during logging: request_path=sys/plugins/catalog error="reflect: reflect.Value.Set using value obtained using unexported field"
 2022-11-30T20:04:22.398Z [ERROR] core: failed to audit response: request_path=sys/plugins/catalog
   error=
   | 1 error occurred:
   | 	* panic generating audit log
   |
 ```

 As a workaround, [listing plugins by type](/vault/api-docs/system/plugins-catalog#list-plugins-1)
 will succeed:

 * `vault list sys/plugins/catalog/auth`
 * `vault list sys/plugins/catalog/database`
 * `vault list sys/plugins/catalog/secret`

 The bug was introduced by commit
 https://github.com/hashicorp/vault/commit/76165052e54f884ed0aa2caa496083dc84ad1c19.

 #### Impacted versions

 Affects versions 1.12.0, 1.12.1, and 1.12.2. A fix will be released in 1.12.3.

 ### PKI OCSP GET requests return malformed request responses

 If an OCSP GET request contains a '+' character, a malformed request response will be
 returned instead of properly processing the request due to a double decoding issue within the
 handler.

 As a workaround, OCSP POST requests can be used which are unaffected.

 #### Impacted versions

 Affects version 1.12.3. A fix will be released in 1.12.4.

 @include 'tokenization-rotation-persistence.mdx'

 @include 'ocsp-redirect.mdx'

 ### LDAP pagination issue

 There was a regression introduced in 1.12.6 relating to LDAP maximum page sizes, resulting in
 an error `no LDAP groups found in groupDN [...] only policies from locally-defined groups available`.  The issue
 occurs when upgrading Vault with an instance that has an existing LDAP Auth configuration.

 As a workaround, disable paged searching using the following:
 ```shell-session
 vault write auth/ldap/config max_page_size=-1
 ```

 #### Impacted versions

 Affects Vault 1.12.6.

 ### Slow startup time when storing PKI certificates

 There was a regression introduced in 1.12.0 where Vault is slow to start because the
 PKI secret engine performs a list operation on the stored certificates. If a large number
 of certificates are stored this can cause long start times on active and standby nodes.

 There is currently no workaround for this other than limiting the number of certificates stored
 in Vault via the [PKI tidy](/vault/api-docs/secret/pki#tidy) or using `no_store`
 flag for [PKI roles](/vault/api-docs/secret/pki#createupdate-role).

 #### Impacted versions

 Affects Vault 1.12.0+

 @include 'pki-double-migration-bug.mdx'
	---
	layout: docs
	page_title: Upgrading to Vault 1.12.x - Guides
	description: \|-
	This page contains the list of deprecations and important or breaking changes
	for Vault 1.12.x. Please read it carefully.
	---

	# Overview

	This page contains the list of deprecations and important or breaking changes
	for Vault 1.12.x compared to 1.11. Please read it carefully.

	## Changes

	### Supported storage backends

	Vault Enterprise will now perform a supported storage check at startup. There is no impact on Vault Community Edition users.

	@include 'ent-supported-storage.mdx'

	@include 'consul-dataplane-upgrade-note.mdx'

	### External plugin loading

	Vault 1.12.0 introduced a change to how external plugins are loaded. Prior to
	Vault 1.12.0 plugins were lazy loaded on startup. This means that plugin
	processes were killed after a successful mount and then respawned when a
	request is routed to them. Vault 1.12.0 introduced auto mutual TLS for
	secrets/auth plugins so we do not lazy load them on startup anymore.

	## Known issues

	### Pinning to builtin plugin versions may cause failure on upgrade

	1.12.0 introduced plugin versions, and with it, the ability to explicitly specify
	the builtin version of a plugin when mounting an auth, database or secrets plugin.
	For example, `vault auth enable -plugin-version=v1.12.0+builtin.vault approle`. If
	there are any mounts where the _builtin_ version was explicitly specified in this way,
	Vault may fail to start on upgrading to 1.12.1 due to the specified version no
	longer being available.

	To check whether a mount path is affected, read the tune information, or the
	database config. The affected plugins are `snowflake-database-plugin@v0.6.0+builtin`
	and any plugins with `+builtin.vault` metadata in their version.

	In this example, the first two mounts are affected because `plugin_version` is
	explicitly set and is one of the affected versions. The third mount is not
	affected because it only has `+builtin` metadata, and is not the Snowflake
	database plugin. All mounts where the version is omitted, or the plugin is
	external (regardless of whether the version is specified) are unaffected.

	-> NOTE: Make sure you use Vault CLI 1.12.0 or later to check mounts.

	```shell-session
	$ vault read sys/auth/approle/tune
	Key Value
	--- -----
	...
	plugin_version v1.12.0+builtin.vault

	$ vault read database/config/snowflake
	Key Value
	--- -----
	...
	plugin_name snowflake-database-plugin
	plugin_version v0.6.0+builtin

	$ vault read sys/auth/kubernetes/tune
	Key Value
	--- -----
	...
	plugin_version v0.14.0+builtin
	```

	As it is not currently possible to unset the plugin version, there are 3 possible
	remediations if you have any affected mounts:

	* Upgrade Vault directly to 1.12.2 once released
	* Upgrade to an external version of the plugin before upgrading to 1.12.1;
	* Using the [tune API](/vault/api-docs/system/auth#tune-auth-method) for auth methods
	* Using the [tune API](/vault/api-docs/system/mounts#tune-mount-configuration) for secrets plugins
	* Or using the [configure connection](/vault/api-docs/secret/databases#configure-connection)
	API for database plugins
	* Unmount and remount the path without a version specified before upgrading to 1.12.1.
	Note: This will delete all data and leases associated with the mount.

	The bug was introduced by commit
	https://github.com/hashicorp/vault/commit/c36330f4c713b886a8a23c08cbbd862a7c530fc8.

	#### Impacted versions

	Affects upgrading from 1.12.0 to 1.12.1. All other upgrade paths are unaffected.
	1.12.2 will introduce a fix that enables upgrades from affected deployments of
	1.12.0.

	### Mounts associated with deprecated builtin plugins will result in core shutdown on upgrade

	As of 1.12.0 Standalone (logical) DB Engines and the AppId Auth Method have been
	marked with the `Pending Removal` status. Any attempt to unseal Vault with
	mounts backed by one of these builtin plugins will result in an immediate
	shutdown of the Vault core.

	-> NOTE In the event that an external plugin with the same name and type as
	a deprecated builtin is deregistered, any subsequent unseal of Vault will also
	result in a core shutdown.

	```shell-session
	$ vault plugin register -sha256=c805cf3b69f704dfcd5176ef1c7599f88adbfd7374e9c76da7f24a32a97abfe1 auth app-id
	Success! Registered plugin: app-id
	$ vault auth enable -plugin-name=app-id plugin
	Success! Enabled app-id auth method at: app-id/
	$ vault auth list -detailed
	app-id/ app-id auth_app-id_3a8f2e24 system system default-service replicated false false map[] n/a 0018263c-0d64-7a70-fd5c-50e05c5f5dc3 n/a n/a c805cf3b69f704dfcd5176ef1c7599f88adbfd7374e9c76da7f24a32a97abfe1 n/a
	$ vault plugin deregister auth app-id
	Success! Deregistered plugin (if it was registered): app-id
	$ vault plugin list -detailed \| grep "app-id"
	app-id auth v1.12.0+builtin.vault pending removal
	```

	The remediation for affected mounts is to set the
	[VAULT_ALLOW_PENDING_REMOVAL_MOUNTS](/vault/docs/commands/server#vault_allow_pending_removal_mounts)
	environment variable and replace any `Pending Removal` feature with the
	[preferred alternative
	feature](/vault/docs/deprecation/faq#q-what-should-i-do-if-i-use-mount-filters-appid-or-any-of-the-standalone-db-engines).

	For more information on the phases of deprecation, see the [Deprecation Notices
	FAQ](/vault/docs/deprecation/faq#q-what-are-the-phases-of-deprecation).

	#### Impacted versions

	Affects upgrading from any version of Vault to 1.12.x. All other upgrade paths
	are unaffected.

	### `vault plugin list` fails when audit logging is enabled

	If audit logging is enabled, Vault will fail to audit the response from any
	calls to the [`GET /v1/sys/plugins/catalog`](/vault/api-docs/system/plugins-catalog#list-plugins)
	endpoint, which causes the whole request to fail and return a 500 internal
	server error. From the CLI, this looks like the following:

	```shell-session
	$ vault plugin list
	Error listing available plugins: data from server response is empty
	```

	It will produce errors in Vault Server's logs such as:

	```text
	2022-11-30T20:04:22.397Z [ERROR] audit: panic during logging: request_path=sys/plugins/catalog error="reflect: reflect.Value.Set using value obtained using unexported field"
	2022-11-30T20:04:22.398Z [ERROR] core: failed to audit response: request_path=sys/plugins/catalog
	error=
	\| 1 error occurred:
	\| * panic generating audit log
	\|
	```

	As a workaround, [listing plugins by type](/vault/api-docs/system/plugins-catalog#list-plugins-1)
	will succeed:

	* `vault list sys/plugins/catalog/auth`
	* `vault list sys/plugins/catalog/database`
	* `vault list sys/plugins/catalog/secret`

	The bug was introduced by commit
	https://github.com/hashicorp/vault/commit/76165052e54f884ed0aa2caa496083dc84ad1c19.

	#### Impacted versions

	Affects versions 1.12.0, 1.12.1, and 1.12.2. A fix will be released in 1.12.3.

	### PKI OCSP GET requests return malformed request responses

	If an OCSP GET request contains a '+' character, a malformed request response will be
	returned instead of properly processing the request due to a double decoding issue within the
	handler.

	As a workaround, OCSP POST requests can be used which are unaffected.

	#### Impacted versions

	Affects version 1.12.3. A fix will be released in 1.12.4.

	@include 'tokenization-rotation-persistence.mdx'

	@include 'ocsp-redirect.mdx'

	### LDAP pagination issue

	There was a regression introduced in 1.12.6 relating to LDAP maximum page sizes, resulting in
	an error `no LDAP groups found in groupDN [...] only policies from locally-defined groups available`. The issue
	occurs when upgrading Vault with an instance that has an existing LDAP Auth configuration.

	As a workaround, disable paged searching using the following:
	```shell-session
	vault write auth/ldap/config max_page_size=-1
	```

	#### Impacted versions

	Affects Vault 1.12.6.

	### Slow startup time when storing PKI certificates

	There was a regression introduced in 1.12.0 where Vault is slow to start because the
	PKI secret engine performs a list operation on the stored certificates. If a large number
	of certificates are stored this can cause long start times on active and standby nodes.

	There is currently no workaround for this other than limiting the number of certificates stored
	in Vault via the [PKI tidy](/vault/api-docs/secret/pki#tidy) or using `no_store`
	flag for [PKI roles](/vault/api-docs/secret/pki#createupdate-role).

	#### Impacted versions

	Affects Vault 1.12.0+

	@include 'pki-double-migration-bug.mdx'