dagster-polars has been added as an integration. Thanks @danielgafni!@dbt_assets now supports loading projects with semantic models.@dbt_assets now supports loading projects with model versions.get_asset_key_for_model now supports retrieving asset keys for seeds and snapshots. Thanks @aksestok!@asset decorator causing a false positive in Pyright strict mode. Thanks @tylershunt!security_context field on the k8s_job_executor didn't correctly set the security context on the launched step pods. Thanks @krgn!ObserveResults with no data_version.FreshnessPolicys on observable source assets. These assets will be considered “Overdue” when the latest value for the “dagster/data_time” metadata value is older than what’s allowed by the freshness policy.DbtCliResource now include metadata from the dbt adapter response. This includes fields like rows_affected, query_id from the Snowflake adapter, or bytes_processed from the BigQuery adapter.k8s_job_executor.context.log.exception in a sensor or schedule did not properly capture exception information.@observable_source_asset-decorated functions can now return an ObserveResult. This allows including metadata on the observation, in addition to a data version. This is currently only supported for non-partitioned assets.AutoMaterializeRule.skip_on_not_all_parents_updated_since_cron class allows you to construct AutoMaterializePolicys which wait for all parents to be updated after the latest tick of a given cron schedule.--name argument is now optional when running dagster project from-example.@asset(key=...).AssetKey now has a with_prefix method.AutoMaterializePolicys with large numbers of partitions.dagster instance migrate now prints information about changes to the instance database schema.dagster-cloud-agent helm chart now supports setting K8s labels on the agent deployment.[ui] Fixed an issue that prevented filtering by date on the job-specific runs tab.
[ui] “F” key with modifiers (alt, ctrl, cmd, shift) no longer toggles the filter menu on pages that support filtering.
[ui] Fix empty states on Runs table view for individual jobs, to provide links to materialize an asset or launch a run for the specific job, instead of linking to global pages.
[ui] When a run is launched from the Launchpad editor while an editor hint popover is open, the popover remained on the page even after navigation. This has been fixed.
[ui] Fixed an issue where clicking on the zoom controls on a DAG view would close the right detail panel for selected nodes.
[ui] Fixed an issue shift-selecting assets with multi-component asset keys.
[ui] Fixed an issue with the truncation of the asset stale causes popover.
When using a TimeWindowPartitionMapping with a start_offset or end_offset specified, requesting the downstream partitions of a given upstream partition would yield incorrect results. This has been fixed.
When using AutoMaterializePolicys with observable source assets, in rare cases, a second run could be launched in response to the same version being observed twice. This has been fixed.
When passing in hook_defs to define_asset_job, if any of those hooks had required resource keys, a missing resource error would surface when the hook was executed. This has been fixed.
Fixed a typo in a documentation URL in dagster-duckdb-polars tests. The URL now works correctly.
DbtManifest to fetch asset keys of sources and models: DbtManifest.get_asset_key_for_model, DbtManifest.get_asset_key_for_source. These methods are utilities for defining python assets as dependencies of dbt assets via @asset(key=manifest.get_asset_key_for_model(...).state_path parameter with DbtManifestAssetSelection has been deprecated, and will be removed in the next minor release.grpcio package (for dagster) has been removed.PartitionMapping have been removed. Defining custom partition mappings has been unsupported since 1.1.7.build_airbyte_assets. Thanks @guy-rvvup!meta.dagster.asset_key. This field takes in a list of strings that are used as the components of the generated AssetKey.version: 2 models: - name: users config: meta: dagster: asset_key: ["my", "custom", "asset_key"]
meta.dagster.group. This field takes in a string that is used as the Dagster group for the generated software-defined asset corresponding to the dbt model.version: 2 models: - name: users config: meta: dagster: group: "my_group"
dagster-msteams and dagster-mlflow packages could be installed with incompatible versions of the dagster package due to a missing pin.dagster-daemon run command sometimes kept code server subprocesses open longer than it needed to, making the process use more memory.@observable_source_assets with AutoMaterializePolicies, it was possible for downstream assets to get “stuck”, not getting materialized when other upstream assets changed, or for multiple down materializations to be kicked off in response to the same version being observed multiple times. This has been fixed.@dbt_assetsproject_dir and target_path in DbtCliTask are converted from type str to type pathlib.Path.stdout.dagster field under +meta configuration. The following are equivalent:Before:
version: 2 models: - name: users config: dagster_freshness_policy: maximum_lag_minutes: 60 cron_schedule: "0 9 * * *" dagster_auto_materialize_policy: type: "lazy"
After:
version: 2 models: - name: users config: meta: dagster: freshness_policy: maximum_lag_minutes: 60 cron_schedule: "0 9 * * *" auto_materialize_policy: type: "lazy"
Added support for Pythonic Config classes to the @configured API, which makes reusing op and asset definitions easier:
class GreetingConfig(Config): message: str @op def greeting_op(config: GreetingConfig): print(config.message) class HelloConfig(Config): name: str @configured(greeting_op) def hello_op(config: HelloConfig): return GreetingConfig(message=f"Hello, {config.name}!")
Added AssetExecutionContext to replace OpExecutionContext as the context object passed in to @asset functions.
TimeWindowPartitionMapping now contains an allow_nonexistent_upstream_partitions argument that, when set to True, allows a downstream partition subset to have nonexistent upstream parents.
Unpinned the alembic dependency in the dagster package.
[ui] A new “Assets” tab is available from the Overview page.
[ui] The Backfills table now includes links to the assets that were targeted by the backfill.
croniter==1.4.0. Users of earlier versions of Dagster can pin croniter<1.4.@run_failure_sensor.py.typed was missing in the dagster-graphql package. Thanks @Tanguy-LeFloch!AutoMaterializePolicys will now be cleared after 1 week.@dbt_assets:profile and target can now be customized on the DbtCli resource.partial_parse.msgpack is detected in the target directory of your dbt project, it is now copied into the target directories created by DbtCli to take advantage of partial parsing.@dbt_assets can now be customized by overriding DbtManifest.node_info_to_metadata.AssetMaterializations.serverK8sConfig.containerConfig.name did not actually change the container name.1.3.8 release where the Dagster Cloud agent would sometimes fail to start up with an import error.define_asset_job now accepts a hooks argument.sqlalchemy==2.xadditionalInstanceConfig key that allows you to supply additional configuration to the Dagster instance.EcsRunLauncher now uses a different task definition family for each job, instead of registering a new task definition revision each time a different job is launched.EcsRunLauncher now includes a run_ecs_tags config key that lets you configure tags on the launched ECS task for each run.SkipReason, the SkipReason would be ignored. This has been fixed.RuntimeError: dictionary changed size during iteration exception while code servers were being reloaded.dagster code-server start CLI, instead of only reloading your code when you initiate a reload from the Dagster UI.metadata parameter to define_asset_job (Thanks Elliot2718!)poll_sync_run to handle the “preparing” status from the Census API (Thanks ldnicolasmay!)@observable_source_asset-decorated functions can now return a DataVersionsByPartition to record versions for partitions.@dbt_assetsDbtCliTask's created by invoking DbtCli.cli(...) now have a method .is_successful(), which returns a boolean representing whether the underlying CLI process executed the dbt command successfully.@dbt_assets can now be customized by overriding DbtManifest.node_info_to_description.@dbt_assets.server_ecs_tags and run_ecs_tags that apply to each service or task created by the agent. See the docs for more information.instance.get_run_partition_data in Dagster Cloud.