Skip to content

Commit

Permalink
Add descriptions for airflow resource config parameters (apache#34438)
Browse files Browse the repository at this point in the history
* Add descriptions for airflow resource config parameters

* Fix failing build doc CI
  • Loading branch information
Owen-CH-Leung authored Sep 17, 2023
1 parent 03db0f6 commit 3a4712e
Showing 1 changed file with 63 additions and 27 deletions.
90 changes: 63 additions & 27 deletions airflow/config_templates/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -756,7 +756,8 @@ logging:
[%%(blue)s%%(asctime)s%%(reset)s] {{%%(blue)s%%(filename)s:%%(reset)s%%(lineno)d}}
%%(log_color)s%%(levelname)s%%(reset)s - %%(log_color)s%%(message)s%%(reset)s
colored_formatter_class:
description: ~
description: |
Specifies the class utilized by Airflow to implement colored logging
version_added: 2.0.0
type: string
example: ~
Expand All @@ -769,7 +770,8 @@ logging:
example: ~
default: "[%%(asctime)s] {{%%(filename)s:%%(lineno)d}} %%(levelname)s - %%(message)s"
simple_log_format:
description: ~
description: |
Defines the format of log messages for simple logging configuration
version_added: 2.0.0
type: string
example: ~
Expand All @@ -790,7 +792,10 @@ logging:
default: "[%%(asctime)s] [SOURCE:DAG_PROCESSOR]
{{%%(filename)s:%%(lineno)d}} %%(levelname)s - %%(message)s"
log_formatter_class:
description: ~
description: |
Determines the formatter class used by Airflow for structuring its log messages
The default formatter class is timezone-aware, which means that timestamps attached to log entries
will be adjusted to reflect the local timezone of the Airflow instance
version_added: 2.3.4
type: string
example: ~
Expand Down Expand Up @@ -954,19 +959,22 @@ metrics:
example: ~
default: "False"
statsd_host:
description: ~
description: |
Specifies the host address where the StatsD daemon (or server) is running
version_added: 2.0.0
type: string
example: ~
default: "localhost"
statsd_port:
description: ~
description: |
Specifies the port on which the StatsD daemon (or server) is listening to
version_added: 2.0.0
type: string
example: ~
default: "8125"
statsd_prefix:
description: ~
description: |
Defines the namespace for all metrics sent from Airflow to StatsD
version_added: 2.0.0
type: string
example: ~
Expand Down Expand Up @@ -1042,19 +1050,24 @@ metrics:
example: ~
default: "localhost"
otel_port:
description: ~
description: |
Specifies the hostname or IP address of the OpenTelemetry Collector to which Airflow sends
metrics and traces
version_added: 2.6.0
type: string
example: ~
default: "8889"
otel_prefix:
description: ~
description: |
Specifies the port of the OpenTelemetry Collector that is listening to
version_added: 2.6.0
type: string
example: ~
default: "airflow"
otel_interval_milliseconds:
description: ~
description: |
Defines the interval, in milliseconds, at which Airflow sends batches of metrics and traces
to the configured OpenTelemetry Collector
version_added: 2.6.0
type: integer
example: ~
Expand Down Expand Up @@ -1281,25 +1294,33 @@ operators:
example: ~
default: "false"
default_cpus:
description: ~
description: |
Indicates the default number of CPU units allocated to each operator when no specific CPU request
is specified in the operator's configuration
version_added: ~
type: string
example: ~
default: "1"
default_ram:
description: ~
description: |
Indicates the default number of RAM allocated to each operator when no specific RAM request
is specified in the operator's configuration
version_added: ~
type: string
example: ~
default: "512"
default_disk:
description: ~
description: |
Indicates the default number of disk storage allocated to each operator when no specific disk request
is specified in the operator's configuration
version_added: ~
type: string
example: ~
default: "512"
default_gpus:
description: ~
description: |
Indicates the default number of GPUs allocated to each operator when no specific GPUs request
is specified in the operator's configuration
version_added: ~
type: string
example: ~
Expand Down Expand Up @@ -1880,56 +1901,66 @@ smtp:
smtp server here
options:
smtp_host:
description: ~
description: |
Specifies the host server address used by Airflow when sending out email notifications via SMTP.
version_added: ~
type: string
example: ~
default: "localhost"
smtp_starttls:
description: ~
description: |
Determines whether to use the STARTTLS command when connecting to the SMTP server.
version_added: ~
type: string
example: ~
default: "True"
smtp_ssl:
description: ~
description: |
Determines whether to use an SSL connection when talking to the SMTP server.
version_added: ~
type: string
example: ~
default: "False"
smtp_user:
description: ~
description: |
Username to authenticate when connecting to smtp server.
version_added: ~
type: string
example: "airflow"
default: ~
smtp_password:
description: ~
description: |
Password to authenticate when connecting to smtp server.
version_added: ~
type: string
sensitive: true
example: "airflow"
default: ~
smtp_port:
description: ~
description: |
Defines the port number on which Airflow connects to the SMTP server to send email notifications.
version_added: ~
type: string
example: ~
default: "25"
smtp_mail_from:
description: ~
description: |
Specifies the default "from" email address used when Airflow sends email notifications.
version_added: ~
type: string
example: ~
default: "[email protected]"
smtp_timeout:
description: ~
description: |
Determines the maximum time (in seconds) the Apache Airflow system will wait for a
connection to the SMTP server to be established.
version_added: 2.0.0
type: integer
example: ~
default: "30"
smtp_retry_limit:
description: ~
description: |
Defines the maximum number of times Airflow will attempt to connect to the SMTP server.
version_added: 2.0.0
type: integer
example: ~
Expand Down Expand Up @@ -2093,7 +2124,8 @@ scheduler:
example: ~
default: "300.0"
child_process_log_directory:
description: ~
description: |
Determines the directory where logs for the child processes of the scheduler will be stored
version_added: ~
type: string
example: ~
Expand Down Expand Up @@ -2319,7 +2351,8 @@ kerberos:
description: ~
options:
ccache:
description: ~
description: |
Location of your ccache file once kinit has been performed.
version_added: ~
type: string
example: ~
Expand All @@ -2332,19 +2365,22 @@ kerberos:
example: ~
default: "airflow"
reinit_frequency:
description: ~
description: |
Determines the frequency at which initialization or re-initialization processes occur.
version_added: ~
type: string
example: ~
default: "3600"
kinit_path:
description: ~
description: |
Path to the kinit executable
version_added: ~
type: string
example: ~
default: "kinit"
keytab:
description: ~
description: |
Designates the path to the Kerberos keytab file for the Airflow user
version_added: ~
type: string
example: ~
Expand Down

0 comments on commit 3a4712e

Please sign in to comment.