Group
Guide to the Secure Configuration of Red Hat OpenShift Container Platform 4
Group contains 7 groups and 15 rules |
Group
OpenShift Settings
Group contains 6 groups and 15 rules |
[ref]
Each section of this configuration guide includes information about the default configuration
of an OpenShift cluster and a set of recommendations for hardening the configuration. For each
hardening recommendation, information on how to implement the control and/or how to verify or audit
the control is provided. In some cases, remediation information is also provided.
Many of the settings in the hardening guide are in place by default. The audit information for these
settings is provided in order to verify that the cluster admininstrator has not made changes that
would be less secure than the OpenShift defaults. A small number of items require configuration.
Finally, there are some recommendations that require decisions by the system operator, such as audit
log size, retention, and related settings. |
Group
OpenShift Kube API Server
Group contains 7 rules |
[ref]
This section contains recommendations for kube-apiserver configuration. |
Rule
Configure the etcd Certificate Authority for the API Server
[ref] | To ensure etcd is configured to make use of TLS encryption for client
connections, follow the OpenShift documentation and setup the TLS
connection between the API Server and etcd. Then, verify
that apiServerArguments has the etcd-cafile configured in
the openshift-kube-apiserver config configmap to something
similar to:
"apiServerArguments": {
...
"etcd-cafile": [
"/etc/kubernetes/static-pod-resources/configmaps/etcd-serving-ca/ca-bundle.crt"
],
...
Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file. | Rationale: | etcd is a highly-available key-value store used by OpenShift deployments
for persistent storage of all REST API objects. These objects are
sensitive in nature and should be protected by client authentication. This
requires the API Server to identify itself to the etcd server using
a SSL Certificate Authority file. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_api_server_etcd_ca | Identifiers and References | Identifiers:
CCE-84216-1 References:
SC-8(1), SC-8(2), 1.2.32 | |
|
Rule
Configure the Encryption Provider Cipher
[ref] | To ensure the correct cipher, set the encryption type aescbc in the
apiserver object which configures the API server itself.
spec:
encryption:
type: aescbc
For more information, follow
the relevant documentation. Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/apiservers/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster file. | Rationale: | aescbc is currently the strongest encryption provider, it should
be preferred over other providers.
| Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_api_server_encryption_provider_cipher | Identifiers and References | References:
SC-28(1), 1.2.34 | |
|
Rule
Configure the Certificate Key for the API Server
[ref] | To ensure the API Server utilizes its own TLS certificates, the
tls-private-key-file must be configured. Verify
that the apiServerArguments section has the tls-private-key-file configured in
the config configmap in the openshift-kube-apiserver namespace
similar to:
"apiServerArguments":{
...
"tls-private-key-file": [
"/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key"
],
...
}
Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file. | Rationale: | API Server communication contains sensitive parameters that should remain
encrypted in transit. Configure the API Server to serve only HTTPS
traffic. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_api_server_tls_private_key | Identifiers and References | Identifiers:
CCE-84282-3 References:
SC-8(1), SC-8(2), 1.2.30 | |
|
Rule
Configure the Client Certificate Authority for the API Server
[ref] | Certificates must be provided to fully setup TLS client certificate
authentication. To ensure the API Server utilizes its own TLS certificates, the
clientCA must be configured. Verify
that servingInfo has the clientCA configured in
the openshift-kube-apiserver config configmap
to something similar to:
"apiServerArguments": {
...
"client-ca-file": [
"/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt"
],
...
Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file. | Rationale: | API Server communication contains sensitive parameters that should remain
encrypted in transit. Configure the API Server to serve only HTTPS traffic.
If -clientCA is set, any request presenting a client
certificate signed by one of the authorities in the client-ca-file
is authenticated with an identity corresponding to the CommonName of
the client certificate. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_api_server_client_ca | Identifiers and References | Identifiers:
CCE-84284-9 References:
SC-8(1), SC-8(2), 1.2.31 | |
|
Rule
Configure the Encryption Provider
[ref] | To encrypt the etcd key-value store, set the encryption type aescbc in the
apiserver object which configures the API server itself.
spec:
encryption:
type: aescbc
For more information, follow
the relevant documentation. Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/apiservers/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster file. | Rationale: | etcd is a highly available key-value store used by OpenShift deployments
for persistent storage of all REST API objects. These objects are
sensitive in nature and should be encrypted at rest to avoid any
disclosures. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_api_server_encryption_provider_config | Identifiers and References | Identifiers:
CCE-83585-0 References:
SC-28(1), 1.2.33 | |
|
Rule
Configure the Certificate for the API Server
[ref] | To ensure the API Server utilizes its own TLS certificates, the
tls-cert-file must be configured. Verify
that the apiServerArguments section has the tls-cert-file configured in
the config configmap in the openshift-kube-apiserver namespace
similar to:
"apiServerArguments":{
...
"tls-cert-file": [
"/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt"
],
...
}
Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file. | Rationale: | API Server communication contains sensitive parameters that should remain
encrypted in transit. Configure the API Server to serve only HTTPS
traffic. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_api_server_tls_cert | Identifiers and References | Identifiers:
CCE-83779-9 References:
SC-8(1), SC-8(2), 1.2.30 | |
|
Rule
Ensure that Audit Log Forwarding Is Enabled
[ref] | OpenShift audit works at the API server level, logging all requests coming to the server.
Audit is on by default and the best practice is to ship audit logs off the cluster for retention.
The cluster-logging-operator is able to do this with the ClusterLogForwarders resource.
The forementioned resource can be configured to logs to different third party systems.
For more information on this, please reference the official documentation:
https://docs.openshift.com/container-platform/4.6/logging/cluster-logging-external.htmlWarning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance API endpoint to the local /kubernetes-api-resources/apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance file. | Rationale: | Retaining logs ensures the ability to go back in time to investigate or correlate any events.
Offloading audit logs from the cluster ensures that an attacker that has access to the cluster will not be able to
tamper with the logs because of the logs being stored off-site. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_audit_log_forwarding_enabled | Identifiers and References | Identifiers:
CCE-84076-9 References:
AC-2(12), AU-9(2), 1.2.23 | |
|
Group
OpenShift Classification Banner
Group contains 1 rule |
[ref]
OpenShift Console Notifications enable creating custom Classification Banners |
Rule
Enable Classification Banner on OpenShift Console
[ref] | A classification banner can be configured by creating a Console Notification CRD on OpenShift Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /apis/console.openshift.io/v1/consolenotifications/classification-banner API endpoint to the local /kubernetes-api-resources/apis/console.openshift.io/v1/consolenotifications/classification-banner file. | Rationale: | Displays to users organization-defined system use notification message or banner before granting access to the system that provides privacy and security notices consistent with applicable federal laws | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_classification_banner | Identifiers and References | References:
AC-8 | |
|
Group
Authentication
Group contains 1 rule |
[ref]
In cloud workloads, there are many ways to create and configure
to multiple authentication services. Some of these authentication
methods by not be secure or common methodologies, or they may not
be secure by default. This section introduces mechanisms for
configuring authentication systems to OpenShift. |
Rule
Do Not Use htpasswd-based IdP
[ref] |
For users to interact with OpenShift Container Platform, they must first
authenticate to the cluster. The authentication layer identifies the user
associated with requests to the OpenShift Container Platform API. The
authorization layer then uses information about the requesting user to
determine if the request is allowed.
Understanding authentication | Authentication | OpenShift Container Platform
The OpenShift Container Platform includes a built-in OAuth server for
token-based authentication. Developers and administrators obtain OAuth
access tokens to authenticate themselves to the API. It is recommended for
an administrator to configure OAuth to specify an identity provider after
the cluster is installed. User access to the cluster is managed through the
identity provider.
Understanding identity provider configuration | Authentication | OpenShift Container Platform
However, not all Identity Providers supported by OpenShift provide the same
level of capabilities. As an example, the htpasswd Identity Provider only
checks the username and password match and provides no means of 2FA, account
lockout or notification mechanism. This rule therefore only allows a subset
of identity providers.
Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/oauths/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/oauths/cluster file. | Rationale: |
With any authentication mechanism the ability to revoke credentials if they
are compromised or no longer required, is a key control. Kubernetes client
certificate authentication does not allow for this due to a lack of support
for certificate revocation.
OpenShift's built-in OAuth server allows credential revocation by relying on
the Identity provider, as well as giving the administrators the ability to
revoke any tokens given to a specific user.
In addition, using an external Identity provider allows for setting up notifications
on account creation or deletion, multi-factor authentication, disabling inactive
accounts or other features required by different compliance standards.
| Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_ocp_idp_no_htpasswd | Identifiers and References | Identifiers:
CCE-84209-6 References:
AC-2(1), AC-2(2), AC-2(3), AC-2(4), AC-2(7), AC-2(8) | |
|
Group
OpenShift - Registry Security Practices
Group contains 2 rules |
[ref]
Contains evaluations for OpenShift registry security practices, and cluster-wide registry configuration. |
Rule
Allowed registries for import are configured
[ref] | The configuration allowedRegistriesForImport limits the container
image registries from which normal users may import images. This is important
to control, as a user who can stand up a malicious registry can then import
content which claims to include the SHAs of legimitate content layers.
You can set the allowed repositories for import by applying the following
manifest using oc patch , e.g. if you save the following snippet to
/tmp/allowed-import-registries-patch.yaml
spec:
allowedRegistriesForImport:
- domainName: my-trusted-registry.internal.example.com
insecure: false
you would call
oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-import-registries-patch.yaml)" --type=merge Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file. | Rationale: | Allowed registries for import should be specified to limit the registries
from which users may import images. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_ocp_allowed_registries_for_import | Identifiers and References | | |
|
Rule
Allowed registries are configured
[ref] | The configuration registrySources.allowedRegistries determines the
permitted registries that the OpenShift container runtime can access for builds
and pods. This configuration setting ensures that all registries other than
those specified are blocked.
You can set the allowed repositories by applying the following manifest using
oc patch , e.g. if you save the following snippet to
/tmp/allowed-registries-patch.yaml
spec:
registrySources:
allowedRegistries:
- my-trusted-registry.internal.example.com
you would call
oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-registries-patch.yaml)" --type=merge Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file. | Rationale: | Allowed registries should be configured to restrict the registries that the
OpenShift container runtime can access, and all other registries should be
blocked. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_ocp_allowed_registries | Identifiers and References | | |
|
Group
OpenShift etcd Settings
Group contains 2 rules |
[ref]
Contains rules that check correct OpenShift etcd settings. |
Rule
Ensure That The etcd Peer Key File Is Correctly Set
[ref] | To ensure the etcd service is serving TLS to peers,
make sure the etcd-pod* ConfigMaps in the
openshift-etcd namespace contain the following argument
for the etcd binary in the etcd pod:
oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-peer-NODE_NAME.key"
Note that the [a-z]+ is being used since the directory might
change between OpenShift versions. Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-etcd/configmaps/etcd-pod API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod file. | Rationale: | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_etcd_peer_key_file | Identifiers and References | Identifiers:
CCE-83711-2 References:
SC-8(1), SC-8(2), 2.4 | |
|
Rule
Ensure That The etcd Peer Client Certificate Is Correctly Set
[ref] | To ensure the etcd service is serving TLS to peers,
make sure the etcd-pod* ConfigMaps in the
openshift-etcd namespace contain the following argument
for the etcd binary in the etcd pod:
--peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-peer-NODE_NAME.crt
Note that the [a-z]+ is being used since the directory might
change between OpenShift versions. Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-etcd/configmaps/etcd-pod API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod file. | Rationale: | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_etcd_peer_cert_file | Identifiers and References | Identifiers:
CCE-83847-4 References:
SC-8(1), SC-8(2), 2.4 | |
|
Group
Kubernetes Kubelet Settings
Group contains 2 rules |
[ref]
The Kubernetes Kubelet is an agent that runs on each node in the cluster. It
makes sure that containers are running in a pod.
The kubelet takes a set of PodSpecs that are provided through various
mechanisms and ensures that the containers described in those PodSpecs are
running and healthy. The kubelet doesn’t manage containers which were not
created by Kubernetes. |
Rule
Ensure That The kubelet Server Key Is Correctly Set
[ref] | To ensure the kubelet TLS private server key certificate is configured, edit the
kubelet configuration file /etc/kubernetes/kubelet.conf
and configure the kubelet private key file.
tlsPrivateKeyFile: /path/to/TLS/private.key Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file. | Rationale: | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_kubelet_configure_tls_key | Identifiers and References | References:
SC-8(1), SC-8(2), 4.2.10 | |
|
Rule
Ensure That The kubelet Client Certificate Is Correctly Set
[ref] | To ensure the kubelet TLS client certificate is configured, edit the
kubelet configuration file /etc/kubernetes/kubelet.conf
and configure the kubelet certificate file.
tlsCertFile: /path/to/TLS/cert.key Warning:
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file. | Rationale: | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | Severity: | medium | Rule ID: | xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cert | Identifiers and References | Identifiers:
CCE-83396-2 References:
SC-8(1), SC-8(2), 4.2.10 | |
|