Secrets

When developing applications that interact with external services, databases, or APIs, it is paramount to handle sensitive credentials securely. Directly embedding values like database passwords, API keys, or cloud access credentials into your code is a significant security risk. The industry best practice is to store such sensitive information externally in a dedicated secret manager (typically in an encrypted form) and then reference these values at runtime, usually via environment variables.

Shoc Platform’s Secret Management

Shoc Platform provides a robust and secure mechanism to define and manage your secrets. A Secret is essentially a key-value pair where the value is a piece of sensitive (or non-sensitive) data you wish to use in your code without hardcoding it.

Secrets can be configured as:

Encrypted: For highly sensitive data, ensuring that the value is stored in an encrypted format.
Non-encrypted: For non-sensitive referential values that you still want to manage externally from your code.

Secret Isolation Levels

Shoc Platform supports two distinct levels of secret isolation to accommodate different sharing requirements:

User Secrets: These secrets are unique to your individual user account. Only you can define, view, and use your user secrets. They are ideal for personal credentials that should not be shared with anyone else.
Workspace Secrets: These secrets are shared across all users within a specific workspace. Any member of the workspace can define, view, and use these secrets. This is perfect for team-wide credentials (e.g., shared database access, common API keys for a project).

Defining Secrets via the Web UI

You can define and manage your secrets conveniently through the Shoc Platform web interface:

Navigate to your desired workspace: Workspace > [Your workspace name].
From the left-hand navigation, select the Secrets section.
Choose either User Secrets or Workspace Secrets to manage secrets at the respective level.

Referencing Secrets in Your Jobs

Once secrets are defined in Shoc Platform, you can easily reference them in your run.shoc.yaml manifest. Shoc Platform will automatically inject these secrets as environment variables into your job’s container at runtime.

To reference secrets, use the env.use field in your run.shoc.yaml:


# ... other run.shoc.yaml configurations ...
env:
  # A list of secret keys to fetch and inject into the job as environment variables.
  use: ['MY_API_KEY', 'DB_PASSWORD']

When the job is being prepared for submission, the Shoc Platform system will look up the secret values associated with the keys MY_API_KEY and DB_PASSWORD. It will then ensure that these values are securely injected into your job’s runtime environment as environment variables, making them accessible to your application code.

Note on Precedence: In cases where you have defined a secret with the same key in both your user secrets and workspace secrets, the user secret will always take precedence and override the workspace secret. This allows individual users to customize credentials for shared keys if needed.

In-Place Overrides for Environment Variables

Shoc Platform also provides an override option within the env section of run.shoc.yaml. This allows you to provide environment variable values directly in your manifest, which can be useful for non-sensitive configuration parameters or for quick testing without needing to define a full secret.


# ... other run.shoc.yaml configurations ...
env:
  # Reference existing secrets (optional, can be combined with override)
  use: ['MY_API_KEY']
  # Provide environment variable values directly in the run manifest.
  override:
    MY_API_KEY: "new-value-for-testing-only" # This will override the secret named MY_API_KEY if it exists
    OTHER_KEY: "a-non-sensitive-config-value"

⚠️

Caution with Overrides for Sensitive Data: While the override option offers flexibility, be extremely cautious when using it to provide sensitive values directly in your run.shoc.yaml. It is highly recommended not to publish files containing sensitive override values (e.g., to a public Git repository). For true secrets, always prefer managing them via the dedicated secret manager and referencing them with env.use.

Overrides have the highest priority when determining the final environment variable value. They will override both user secrets and workspace secrets if a key conflict occurs. We recommend using override for providing non-sensitive arguments to your job to avoid embedding them into your code. This approach also prevents unnecessary package rebuilds, as changing an override value in run.shoc.yaml does not require a new package build, unlike changing values directly within your source code.

Accessing Secrets via CLI

You can also list your configured secrets directly from the Shoc CLI:

To list your user secrets:
```
shoc user-secrets list
```
To list your workspace secrets:
```
shoc workspace-secrets list
```