Networks ¶

Roadway Network Format ¶

RoadwayNetworks must be defined in the following format, which leverages Open Street Map which uses a tag-based format (e.g. json, xml). This may be changed in the future to align with the tabular-based General Modeling Network Specification.

A network is defined by a set of nodes and links which connect them. Shapes may be optionally specified for each link in a separate file.

file serialization formats

While the default serialiation for roadway networks is json/geojson and for transit data is csv, networks can also be stored – more efficiently – in parquet files with a similar structure. Other workable file serializations include shapefiles, csvs, and anything that can be read by pandas or geopandas. This can be noted in most I/O procedures by including the keyword argument file_format = <format>.

Roadway Validation ¶

RoadwayNetworks can be validated using the following tools:

CLIPython API

python validate_roadway.py <network_directory> <file_format> [-s] [--output_dir <output_dir>]

Where:

network_directory: The roadway network file directory.
file_format: The suffices of roadway network file name.
-s, --strict: Validate the roadway network strictly without parsing and filling in data.
--output_dir: The output directory for the validation report.

from network_wrangler.roadway.validate import validate_roadway_in_dir
validate_roadway_in_dir(
    directory=<network_directory>,
    file_format=<file_format>,
    strict=<strict_bool>>,
    output_dir=<output_dir>
)

Where:

network_directory: The roadway network file directory.
file_format: The suffices of roadway network file name.
strict: Validate the roadway network strictly without parsing and filling in data.
output_dir: The output directory for the validation report.

Examples ¶

Network Wrangler is packaged with two examples located in the /examples directory:

St Paul, MN
Small which is a several block exerpt of St Paul and is infinitely easier to trouble-shoot quickly.

Road Nodes ¶

A valid geojson, shp, or parquet file.

Datamodel used to validate if links_df is of correct format and types.

Must have a record for each node used by the links table and by the transit shapes, stop_times, and stops tables.

Attributes:

Name	Type	Description
`model_node_id`	`int`	Unique identifier for the node.
`osm_node_id`	`Optional[str]`	Reference to open street map node id. Used for querying. Not guaranteed to be unique.
`X`	`float`	Longitude of the node in WGS84. Must be in the range of -180 to 180.
`Y`	`float`	Latitude of the node in WGS84. Must be in the range of -90 to 90.
`geometry`	`GeoSeries`	Warning: this attribute is controlled by wrangler and should not be explicitly user-edited.

Source code in network_wrangler/models/roadway/tables.py

class RoadNodesTable(DataFrameModel):
    """Datamodel used to validate if links_df is of correct format and types.

    Must have a record for each node used by the `links` table and by the transit `shapes`, `stop_times`, and `stops` tables.

    Attributes:
        model_node_id (int): Unique identifier for the node.
        osm_node_id (Optional[str]): Reference to open street map node id. Used for querying. Not guaranteed to be unique.
        X (float): Longitude of the node in WGS84. Must be in the range of -180 to 180.
        Y (float): Latitude of the node in WGS84. Must be in the range of -90 to 90.
        geometry (GeoSeries): **Warning**: this attribute is controlled by wrangler and should not be explicitly user-edited.
    """

    model_node_id: Series[int] = pa.Field(coerce=True, unique=True, nullable=False)
    model_node_idx: Optional[Series[int]] = pa.Field(coerce=True, unique=True, nullable=False)
    X: Series[float] = pa.Field(coerce=True, nullable=False)
    Y: Series[float] = pa.Field(coerce=True, nullable=False)
    geometry: GeoSeries

    # optional fields
    osm_node_id: Series[str] = pa.Field(
        coerce=True,
        nullable=True,
        default="",
    )
    projects: Series[str] = pa.Field(coerce=True, default="")
    inboundReferenceIds: Optional[Series[list[str]]] = pa.Field(coerce=True, nullable=True)
    outboundReferenceIds: Optional[Series[list[str]]] = pa.Field(coerce=True, nullable=True)

    class Config:
        """Config for RoadNodesTable."""

        add_missing_columns = True
        coerce = True
        _pk: ClassVar[TablePrimaryKeys] = ["model_node_id"]

Road Links ¶

A valid geojson, shp, parquet, or json file.

Datamodel used to validate if links_df is of correct format and types.

Attributes:

Name	Type	Description
`model_link_id`	`int`	Unique identifier for the link.
`A`	`int`	`model_node_id` of the link’s start node. Foreign key to `road_nodes`.
`B`	`int`	`model_node_id` of the link’s end node. Foreign key to `road_nodes`.
`geometry`	`GeoSeries`	Warning: this attribute is controlled by wrangler and should not be explicitly user-edited. Simple A→B geometry of the link.
`name`	`str`	Name of the link.
`rail_only`	`bool`	If the link is only for rail. Default is False.
`bus_only`	`bool`	If the link is only for buses. Default is False.
`drive_access`	`bool`	If the link allows driving. Default is True.
`bike_access`	`bool`	If the link allows biking. Default is True.
`walk_access`	`bool`	If the link allows walking. Default is True.
`truck_access`	`bool`	If the link allows trucks. Default is True.
`distance`	`float`	Length of the link.
`roadway`	`str`	Type of roadway per OSM definitions. Default is “road”.
`projects`	`str`	Warning: this attribute is controlled by wrangler and should not be explicitly user-edited. Comma-separated list of project names applied to the link. Default is “”.
`managed`	`int`	Warning: this attribute is controlled by wrangler and should not be explicitly user-edited. Indicator for the type of managed lane facility. Values can be: 0 indicating no managed lane on this link. 1 indicates that there is a managed lane on the link (std network) or that the link is a managed lane (model network). -1 indicates that there is a parallel managed lane derived from this link (model network).
`shape_id`	`str`	Identifier referencing the primary key of the shapes table. Default is None.
`lanes`	`int`	Default number of lanes on the link. Default is 1.
`sc_lanes`	`Optional[list[dict]]`	List of scoped link values for the number of lanes. Default is None. Example: `[{'timespan':['12:00':'15:00'], 'value': 3},{'timespan':['15:00':'19:00'], 'value': 2}]`.
`price`	`float`	Default price to use the link. Default is 0.
`sc_price`	`Optional[list[dict]]`	List of scoped link values for the price. Default is None. Example: `[{'timespan':['15:00':'19:00'],'category': 'sov', 'value': 2.5}]`.
`ref`	`Optional[str]`	Reference numbers for link referring to a route or exit number per the OSM definition. Default is None.
`access`	`Optional[Any]`	User-defined method to note access restrictions for the link. Default is None.
`ML_projects`	`Optional[str]`	Warning: this attribute is controlled by wrangler and should not be explicitly user-edited. Comma-separated list of project names applied to the managed lane. Default is “”.
`ML_lanes`	`Optional[int]`	Default number of lanes on the managed lane. Default is None.
`ML_price`	`Optional[float]`	Default price to use the managed lane. Default is 0.
`ML_access`	`Optional[Any]`	User-defined method to note access restrictions for the managed lane. Default is None.
`ML_access_point`	`Optional[bool]`	If the link is an access point for the managed lane. Default is False.
`ML_egress_point`	`Optional[bool]`	If the link is an egress point for the managed lane. Default is False.
`sc_ML_lanes`	`Optional[list[dict]]`	List of scoped link values for the number of lanes on the managed lane. Default is None.
`sc_ML_price`	`Optional[list[dict]]`	List of scoped link values for the price of the managed lane. Default is None.
`sc_ML_access`	`Optional[list[dict]]`	List of scoped link values for the access restrictions of the managed lane. Default is None.
`ML_geometry`	`Optional[GeoSeries]`	Warning: this attribute is controlled by wrangler and should not be explicitly user-edited. Simple A→B geometry of the managed lane. Default is None.
`ML_shape_id`	`Optional[str]`	Identifier referencing the primary key of the shapes table for the managed lane. Default is None.
`osm_link_id`	`Optional[str]`	Identifier referencing the OSM link ID. Default is “”.
`GP_A`	`Optional[int]`	Warning: this attribute is controlled by wrangler and should not be explicitly user-edited. Identifier referencing the primary key of the associated general purpose link start node for a managed lane link in a model network. Default is None.
`GP_B`	`Optional[int]`	Warning: this attribute is controlled by wrangler and should not be explicitly user-edited. Identifier referencing the primary key of the associated general purpose link end node for a managed lane link in a model network. Default is None.

User Defined Properties

Additional properites may be defined and are assumed to have the same definition of OpenStreetMap if they have overlapping property names.

Properties for parallel managed lanes ¶

Properties for parallel managed lanes are prefixed with ML_. (Almost) any property, including an ad-hoc one, can be made to apply to a parallel managed lane by applying the prefix ML_, e.g. ML_lanes

Warning

The following properties should not be assigned an ML_ prefix by the user because they are assigned one within networkwrangler:

name
A
B
model_link_id

Time- or category-dependent properties ¶

The following properties can be time-dependent, category-dependent, or both by adding sc_. The “plain” property without the prefix becomes the default when no scoped property applies.

Property	# of Lanes	Price
Default value	`lanes`	`price`
Time- and/or category-dependent value	`sc_lanes`	`sc_price`
Default value for managed lane	`ML_lanes`	`ML_price`
Time- and/or category-dependent value for managed lane	`sc_ML_lanes`	`sc_ML_price`

previous format for scoped properties

Some previous tooling was developed around a previous method for serializing scoped properties. In order to retain compatability with this format:

load_roadway_from_dir(), read_links(), and associated functions will “sniff” the network for the old format and apply the converter function translate_links_df_v0_to_v1()
write_links() has an boolean attribute to convert_complex_properties_to_single_field which can also be invoked from write_roadway() as convert_complex_link_properties_to_single_field.

Defining time-dependent properties ¶

Time-dependent properties are defined as a list of dictionaries with timespans and values.

Timespans must be defined as a list of HH:MM or HH:MM:SS using a 24-hour clock: ('06:00':'09:00').
Timespans must not intersect.

Time-dependent property

$3 peak-period pricing

# default price
'price' = 0
'sc_price':
[
    {
        'time':['06:00':'09:00'],
        'value': 3
    },
    {
        'timespan':['16:00':'19:00'],
        'value': 3,
    }
]

Defining time- and category-dependent properties ¶

Properties co-dependent on time- and category are defined as a list of dictionaries with value, category and time defined.

time- and category-dependent property

A pricing strategy which only applies in peak period for trucks and sovs:

# default price
"price": 0
# price scoped by time of day
"sc_price":
[
    {
        'timespan':['06:00':'09:00'],
        'category': ('sov','truck'),
        'value': 3
    },
    {
        'timespan':['16:00':'19:00'],
        'category': ('sov','truck'),
        'value': 3,
    }
]

Tip

There is no limit on other, user-defined properties being listed as time-dependent or time- and category-dependent.

User-defined variable by time of day

Define a variable access to represent which categories can access the network and vary it by time of day.

#access
{
    # default value for access
    'access': ('any'),
    # scoped value for access
    'sc_access': [
        {
            'timespan':['06:00':'09:00'],
            'value': ('no-trucks')
        },
        {
            'timespan':['16:00':'19:00'],
            'value': ('hov2','hov3','trucks')
        }
    ]
}

Source code in network_wrangler/models/roadway/tables.py

class RoadLinksTable(DataFrameModel):
    """Datamodel used to validate if links_df is of correct format and types.

    Attributes:
        model_link_id (int): Unique identifier for the link.
        A (int): `model_node_id` of the link's start node. Foreign key to `road_nodes`.
        B (int): `model_node_id` of the link's end node. Foreign key to `road_nodes`.
        geometry (GeoSeries): **Warning**: this attribute is controlled by wrangler and should not be explicitly user-edited.
            Simple A-->B geometry of the link.
        name (str): Name of the link.
        rail_only (bool): If the link is only for rail. Default is False.
        bus_only (bool): If the link is only for buses. Default is False.
        drive_access (bool): If the link allows driving. Default is True.
        bike_access (bool): If the link allows biking. Default is True.
        walk_access (bool): If the link allows walking. Default is True.
        truck_access (bool): If the link allows trucks. Default is True.
        distance (float): Length of the link.
        roadway (str): Type of roadway per [OSM definitions](https://wiki.openstreetmap.org/wiki/Key:highway#Roads).
            Default is "road".
        projects (str): **Warning**: this attribute is controlled by wrangler and should not be explicitly user-edited.
            Comma-separated list of project names applied to the link. Default is "".
        managed (int): **Warning**: this attribute is controlled by wrangler and should not be explicitly user-edited.
            Indicator for the type of managed lane facility. Values can be:

            - 0 indicating no managed lane on this link.
            - 1 indicates that there is a managed lane on the link (std network) or that the link is a
                managed lane (model network).
            - -1 indicates that there is a parallel managed lane derived from this link (model network).
        shape_id (str): Identifier referencing the primary key of the shapes table. Default is None.
        lanes (int): Default number of lanes on the link. Default is 1.
        sc_lanes (Optional[list[dict]]: List of scoped link values for the number of lanes. Default is None.
            Example: `[{'timespan':['12:00':'15:00'], 'value': 3},{'timespan':['15:00':'19:00'], 'value': 2}]`.

        price (float): Default price to use the link. Default is 0.
        sc_price (Optional[list[dict]]): List of scoped link values for the price. Default is None.
            Example: `[{'timespan':['15:00':'19:00'],'category': 'sov', 'value': 2.5}]`.
        ref (Optional[str]): Reference numbers for link referring to a route or exit number per the
            [OSM definition](https://wiki.openstreetmap.org/wiki/Key:ref). Default is None.
        access (Optional[Any]): User-defined method to note access restrictions for the link. Default is None.
        ML_projects (Optional[str]): **Warning**: this attribute is controlled by wrangler and should not be explicitly user-edited.
            Comma-separated list of project names applied to the managed lane. Default is "".
        ML_lanes (Optional[int]): Default number of lanes on the managed lane. Default is None.
        ML_price (Optional[float]): Default price to use the managed lane. Default is 0.
        ML_access (Optional[Any]): User-defined method to note access restrictions for the managed lane. Default is None.
        ML_access_point (Optional[bool]): If the link is an access point for the managed lane. Default is False.
        ML_egress_point (Optional[bool]): If the link is an egress point for the managed lane. Default is False.
        sc_ML_lanes (Optional[list[dict]]): List of scoped link values for the number of lanes on the managed lane.
            Default is None.
        sc_ML_price (Optional[list[dict]]): List of scoped link values for the price of the managed lane. Default is None.
        sc_ML_access (Optional[list[dict]]): List of scoped link values for the access restrictions of the managed lane.
            Default is None.
        ML_geometry (Optional[GeoSeries]): **Warning**: this attribute is controlled by wrangler and should not be explicitly user-edited.
            Simple A-->B geometry of the managed lane. Default is None.
        ML_shape_id (Optional[str]): Identifier referencing the primary key of the shapes table for the managed lane.
            Default is None.
        osm_link_id (Optional[str]): Identifier referencing the OSM link ID. Default is "".
        GP_A (Optional[int]): **Warning**: this attribute is controlled by wrangler and should not be explicitly user-edited.
            Identifier referencing the primary key of the associated general purpose link start node for
            a managed lane link in a model network. Default is None.
        GP_B (Optional[int]): **Warning**: this attribute is controlled by wrangler and should not be explicitly user-edited.
            Identifier referencing the primary key of the associated general purpose link end node for
            a managed lane link in a model network. Default is None.

    !!! tip "User Defined Properties"

        Additional properites may be defined and are assumed to have the same definition of OpenStreetMap if they
        have overlapping property names.

    ### Properties for parallel managed lanes

    Properties for parallel managed lanes are prefixed with `ML_`. (Almost) any property,
    including an ad-hoc one, can be made to apply to a parallel managed lane by applying
    the prefix `ML_`, e.g. `ML_lanes`

    !!! warning

        The following properties should **not** be assigned an `ML_` prefix by the user
        because they are assigned one within networkwrangler:

        - `name`
        - `A`
        - `B`
        - `model_link_id`

    ### Time- or category-dependent properties

    The following properties can be time-dependent, category-dependent, or both by adding `sc_`.
    The "plain" property without the prefix becomes the default when no scoped property applies.

    | Property | # of Lanes | Price |
    | -----------| ----------------- | ---------------- |
    | Default value | `lanes` | `price` |
    | Time- and/or category-dependent value | `sc_lanes` | `sc_price` |
    | Default value for managed lane | `ML_lanes` | `ML_price` |
    | Time- and/or category-dependent value for managed lane | `sc_ML_lanes` | `sc_ML_price` |


    ??? note "previous format for scoped properties"

        Some previous tooling was developed around a previous method for serializing scoped properties.  In order to retain compatability with this format:

        - `load_roadway_from_dir()`, `read_links()`, and associated functions will "sniff" the network for the old format and apply the converter function `translate_links_df_v0_to_v1()`
        - `write_links()` has an boolean attribute to `convert_complex_properties_to_single_field` which can also be invoked from `write_roadway()` as `convert_complex_link_properties_to_single_field`.

    #### Defining time-dependent properties

    Time-dependent properties are defined as a list of dictionaries with timespans and values.

    - Timespans must be defined as a list of HH:MM or HH:MM:SS using a 24-hour clock: `('06:00':'09:00')`.
    - Timespans must not intersect.

    !!! example  "Time-dependent property"

        $3 peak-period pricing

        ```python
        # default price
        'price' = 0
        'sc_price':
        [
            {
                'time':['06:00':'09:00'],
                'value': 3
            },
            {
                'timespan':['16:00':'19:00'],
                'value': 3,
            }
        ]
        ```

    #### Defining time- and category-dependent properties

    Properties co-dependent on time- and category are defined as a list of dictionaries with value, category and time defined.

    !!! example "time- and category-dependent property"

        A pricing strategy which only applies in peak period for trucks and sovs:

        ```python
        # default price
        "price": 0
        # price scoped by time of day
        "sc_price":
        [
            {
                'timespan':['06:00':'09:00'],
                'category': ('sov','truck'),
                'value': 3
            },
            {
                'timespan':['16:00':'19:00'],
                'category': ('sov','truck'),
                'value': 3,
            }
        ]
        ```

    !!! tip

        There is no limit on other, user-defined properties being listed as time-dependent or time- and category-dependent.

    !!! example "User-defined variable by time of day"

        Define a variable `access` to represent which categories can access the network and vary it by time of day.

        ```python
        #access
        {
            # default value for access
            'access': ('any'),
            # scoped value for access
            'sc_access': [
                {
                    'timespan':['06:00':'09:00'],
                    'value': ('no-trucks')
                },
                {
                    'timespan':['16:00':'19:00'],
                    'value': ('hov2','hov3','trucks')
                }
            ]
        }
        ```
    """

    model_link_id: Series[int] = pa.Field(coerce=True, unique=True)
    model_link_id_idx: Optional[Series[int]] = pa.Field(coerce=True, unique=True)
    A: Series[int] = pa.Field(nullable=False, coerce=True)
    B: Series[int] = pa.Field(nullable=False, coerce=True)
    geometry: GeoSeries = pa.Field(nullable=False)
    name: Series[str] = pa.Field(nullable=False, default="unknown")
    rail_only: Series[bool] = pa.Field(coerce=True, nullable=False, default=False)
    bus_only: Series[bool] = pa.Field(coerce=True, nullable=False, default=False)
    drive_access: Series[bool] = pa.Field(coerce=True, nullable=False, default=True)
    bike_access: Series[bool] = pa.Field(coerce=True, nullable=False, default=True)
    walk_access: Series[bool] = pa.Field(coerce=True, nullable=False, default=True)
    distance: Series[float] = pa.Field(coerce=True, nullable=False)

    roadway: Series[str] = pa.Field(nullable=False, default="road")
    projects: Series[str] = pa.Field(coerce=True, default="")
    managed: Series[int] = pa.Field(coerce=True, nullable=False, default=0)

    shape_id: Series[str] = pa.Field(coerce=True, nullable=True)
    lanes: Series[int] = pa.Field(coerce=True, nullable=False)
    price: Series[float] = pa.Field(coerce=True, nullable=False, default=0)

    # Optional Fields
    ref: Optional[Series[str]] = pa.Field(coerce=True, nullable=True, default=None)
    access: Optional[Series[Any]] = pa.Field(coerce=True, nullable=True, default=None)

    sc_lanes: Optional[Series[object]] = pa.Field(coerce=True, nullable=True, default=None)
    sc_price: Optional[Series[object]] = pa.Field(coerce=True, nullable=True, default=None)

    ML_projects: Series[str] = pa.Field(coerce=True, default="")
    ML_lanes: Optional[Series[Int64]] = pa.Field(coerce=True, nullable=True, default=None)
    ML_price: Optional[Series[float]] = pa.Field(coerce=True, nullable=True, default=0)
    ML_access: Optional[Series[Any]] = pa.Field(coerce=True, nullable=True, default=True)
    ML_access_point: Optional[Series[bool]] = pa.Field(
        coerce=True,
        default=False,
    )
    ML_egress_point: Optional[Series[bool]] = pa.Field(
        coerce=True,
        default=False,
    )
    sc_ML_lanes: Optional[Series[object]] = pa.Field(
        coerce=True,
        nullable=True,
        default=None,
    )
    sc_ML_price: Optional[Series[object]] = pa.Field(
        coerce=True,
        nullable=True,
        default=None,
    )
    sc_ML_access: Optional[Series[object]] = pa.Field(
        coerce=True,
        nullable=True,
        default=None,
    )

    ML_geometry: Optional[GeoSeries] = pa.Field(nullable=True, coerce=True, default=None)
    ML_shape_id: Optional[Series[str]] = pa.Field(nullable=True, coerce=True, default=None)

    truck_access: Optional[Series[bool]] = pa.Field(coerce=True, nullable=True, default=True)
    osm_link_id: Series[str] = pa.Field(coerce=True, nullable=True, default="")
    # todo this should be List[dict] but ranch output something else so had to have it be Any.
    locationReferences: Optional[Series[Any]] = pa.Field(
        coerce=True,
        nullable=True,
        default="",
    )

    GP_A: Optional[Series[Int64]] = pa.Field(coerce=True, nullable=True, default=None)
    GP_B: Optional[Series[Int64]] = pa.Field(coerce=True, nullable=True, default=None)

    class Config:
        """Config for RoadLinksTable."""

        add_missing_columns = True
        coerce = True
        unique: ClassVar[list[str]] = ["A", "B"]

    @pa.check("sc_*", regex=True, element_wise=True)
    def check_scoped_fields(cls, scoped_value: Series) -> Series[bool]:
        """Checks that all fields starting with 'sc_' or 'sc_ML_' are valid ScopedLinkValueList.

        Custom check to validate fields starting with 'sc_' or 'sc_ML_'
        against a ScopedLinkValueItem model, handling both mandatory and optional fields.
        """
        if scoped_value is None or (not isinstance(scoped_value, list) and pd.isna(scoped_value)):
            return True
        return validate_pyd(scoped_value, ScopedLinkValueList)

Road Shapes ¶

A valid geojson, shp, or parquet file with LineString geometry features and the folowing properties.

Datamodel used to validate if shapes_df is of correct format and types.

Should have a record for each shape_id referenced in links table.

Attributes:

Name	Type	Description
`shape_id`	`str`	Unique identifier for the shape.
`geometry`	`GeoSeries`	Warning: this attribute is controlled by wrangler and should not be explicitly user-edited. Geometry of the shape.
`ref_shape_id`	`Optional[str]`	Reference to another `shape_id` that it may have been created from. Default is None.

Source code in network_wrangler/models/roadway/tables.py

class RoadShapesTable(DataFrameModel):
    """Datamodel used to validate if shapes_df is of correct format and types.

    Should have a record for each `shape_id` referenced in `links` table.

    Attributes:
        shape_id (str): Unique identifier for the shape.
        geometry (GeoSeries): **Warning**: this attribute is controlled by wrangler and should not be explicitly user-edited.
            Geometry of the shape.
        ref_shape_id (Optional[str]): Reference to another `shape_id` that it may
            have been created from. Default is None.
    """

    shape_id: Series[str] = pa.Field(unique=True)
    shape_id_idx: Optional[Series[int]] = pa.Field(unique=True)

    geometry: GeoSeries = pa.Field()
    ref_shape_id: Optional[Series] = pa.Field(nullable=True)

    class Config:
        """Config for RoadShapesTable."""

        coerce = True
        _pk: ClassVar[TablePrimaryKeys] = ["shape_id"]

Transit Network Format ¶

Data models for various GTFS tables using pandera library.

The module includes the following classes:

AgencyTable: Optional. Represents the Agency table in the GTFS dataset.
WranglerStopsTable: Represents the Stops table in the GTFS dataset.
RoutesTable: Represents the Routes table in the GTFS dataset.
WranglerShapesTable: Represents the Shapes table in the GTFS dataset.
WranglerStopTimesTable: Represents the Stop Times table in the GTFS dataset.
WranglerTripsTable: Represents the Trips table in the GTFS dataset.

Each table model leverages the Pydantic data models defined in the records module to define the data model for the corresponding table. The classes also include additional configurations for, such as uniqueness constraints.

Validating a table to the WranglerStopsTable

from network_wrangler.models.gtfs.tables import WranglerStopsTable
from network_wrangler.utils.modesl import validate_df_to_model

validated_stops_df = validate_df_to_model(stops_df, WranglerStopsTable)

Transit Networks must use the the GTFS Schedule format with the following additional constraints:

At this time, only frequency-based schedules are supported.
Each stop_id must be a node in the RoadwayNetwork.
shapes.txt is required (it is optional in GTFS) and must have the added field model_node_id associating a specific location with a node on the RoadwayNetwork.

Stops ¶

Wrangler flavor of GTFS StopsTable.

For field definitions, see the GTFS reference: https://gtfs.org/documentation/schedule/reference/#stopstxt

Attributes:

Name	Type	Description
`stop_id`	`int`	The stop_id. Primary key. Required to be unique. Wrangler assumes that this is a reference to a roadway node and as such must be an integer
`stop_lat`	`float`	The stop latitude.
`stop_lon`	`float`	The stop longitude.
`wheelchair_boarding`	`Optional[int]`	The wheelchair boarding.
`stop_code`	`Optional[str]`	The stop code.
`stop_name`	`Optional[str]`	The stop name.
`tts_stop_name`	`Optional[str]`	The text-to-speech stop name.
`stop_desc`	`Optional[str]`	The stop description.
`zone_id`	`Optional[str]`	The zone id.
`stop_url`	`Optional[str]`	The stop URL.
`location_type`	`Optional[LocationType]`	The location type. Values can be: - 0: stop platform - 1: station - 2: entrance/exit - 3: generic node - 4: boarding area Default of blank assumes a stop platform.
`parent_station`	`Optional[int]`	The `stop_id` of the parent station. Since stop_id is an integer in Wrangler, this field is also an integer
`stop_timezone`	`Optional[str]`	The stop timezone.
`stop_id_GTFS`	`Optional[str]`	The stop_id from the GTFS data.
`projects`	`str`	A comma-separated string value for projects that have been applied to this stop.

Source code in network_wrangler/models/gtfs/tables.py

class WranglerStopsTable(StopsTable):
    """Wrangler flavor of GTFS StopsTable.

    For field definitions, see the GTFS reference: <https://gtfs.org/documentation/schedule/reference/#stopstxt>

    Attributes:
        stop_id (int): The stop_id. Primary key. Required to be unique. **Wrangler assumes that this is a reference to a roadway node and as such must be an integer**
        stop_lat (float): The stop latitude.
        stop_lon (float): The stop longitude.
        wheelchair_boarding (Optional[int]): The wheelchair boarding.
        stop_code (Optional[str]): The stop code.
        stop_name (Optional[str]): The stop name.
        tts_stop_name (Optional[str]): The text-to-speech stop name.
        stop_desc (Optional[str]): The stop description.
        zone_id (Optional[str]): The zone id.
        stop_url (Optional[str]): The stop URL.
        location_type (Optional[LocationType]): The location type. Values can be:
            - 0: stop platform
            - 1: station
            - 2: entrance/exit
            - 3: generic node
            - 4: boarding area
            Default of blank assumes a stop platform.
        parent_station (Optional[int]): The `stop_id` of the parent station. **Since stop_id is an integer in Wrangler, this field is also an integer**
        stop_timezone (Optional[str]): The stop timezone.
        stop_id_GTFS (Optional[str]): The stop_id from the GTFS data.
        projects (str): A comma-separated string value for projects that have been applied to this stop.
    """

    stop_id: Series[int] = pa.Field(
        coerce=True, nullable=False, unique=True, description="The model_node_id."
    )
    stop_id_GTFS: Series[str] = pa.Field(
        coerce=True,
        nullable=True,
        description="The stop_id from the GTFS data",
    )
    stop_lat: Series[float] = pa.Field(coerce=True, nullable=True, ge=-90, le=90)
    stop_lon: Series[float] = pa.Field(coerce=True, nullable=True, ge=-180, le=180)
    projects: Series[str] = pa.Field(coerce=True, default="")

Routes ¶

Represents the Routes table in the GTFS dataset.

For field definitions, see the GTFS reference: https://gtfs.org/documentation/schedule/reference/#routestxt

Attributes:

Name	Type	Description
`route_id`	`str`	The route_id. Primary key. Required to be unique.
`route_short_name`	`Optional[str]`	The route short name.
`route_long_name`	`Optional[str]`	The route long name.
`route_type`	`RouteType`	The route type. Required. Values can be: - 0: Tram, Streetcar, Light rail - 1: Subway, Metro - 2: Rail - 3: Bus
`agency_id`	`Optional[str]`	The agency_id. Foreign key to agency_id in the agencies table.
`route_desc`	`Optional[str]`	The route description.
`route_url`	`Optional[str]`	The route URL.
`route_color`	`Optional[str]`	The route color.
`route_text_color`	`Optional[str]`	The route text color.

Source code in network_wrangler/models/gtfs/tables.py

class RoutesTable(pa.DataFrameModel):
    """Represents the Routes table in the GTFS dataset.

    For field definitions, see the GTFS reference: <https://gtfs.org/documentation/schedule/reference/#routestxt>

    Attributes:
        route_id (str): The route_id. Primary key. Required to be unique.
        route_short_name (Optional[str]): The route short name.
        route_long_name (Optional[str]): The route long name.
        route_type (RouteType): The route type. Required. Values can be:
            - 0: Tram, Streetcar, Light rail
            - 1: Subway, Metro
            - 2: Rail
            - 3: Bus
        agency_id (Optional[str]): The agency_id. Foreign key to agency_id in the agencies table.
        route_desc (Optional[str]): The route description.
        route_url (Optional[str]): The route URL.
        route_color (Optional[str]): The route color.
        route_text_color (Optional[str]): The route text color.
    """

    route_id: Series[str] = pa.Field(nullable=False, unique=True, coerce=True)
    route_short_name: Series[str] = pa.Field(nullable=True, coerce=True)
    route_long_name: Series[str] = pa.Field(nullable=True, coerce=True)
    route_type: Series[Category] = pa.Field(
        dtype_kwargs={"categories": RouteType}, coerce=True, nullable=False
    )

    # Optional Fields
    agency_id: Optional[Series[str]] = pa.Field(nullable=True, coerce=True)
    route_desc: Optional[Series[str]] = pa.Field(nullable=True, coerce=True)
    route_url: Optional[Series[str]] = pa.Field(nullable=True, coerce=True)
    route_color: Optional[Series[str]] = pa.Field(nullable=True, coerce=True)
    route_text_color: Optional[Series[str]] = pa.Field(nullable=True, coerce=True)

    class Config:
        """Config for the RoutesTable data model."""

        coerce = True
        add_missing_columns = True
        _pk: ClassVar[TablePrimaryKeys] = ["route_id"]
        _fk: ClassVar[TableForeignKeys] = {"agency_id": ("agencies", "agency_id")}

Trips ¶

Represents the Trips table in the Wrangler feed, adding projects list.

For field definitions, see the GTFS reference: https://gtfs.org/documentation/schedule/reference/#tripstxt

Attributes:

Name	Type	Description
`trip_id`	`str`	Primary key. Required to be unique.
`shape_id`	`str`	Foreign key to `shape_id` in the shapes table.
`direction_id`	`DirectionID`	The direction id. Required. Values can be: - 0: Outbound - 1: Inbound
`service_id`	`str`	The service id.
`route_id`	`str`	The route id. Foreign key to `route_id` in the routes table.
`trip_short_name`	`Optional[str]`	The trip short name.
`trip_headsign`	`Optional[str]`	The trip headsign.
`block_id`	`Optional[str]`	The block id.
`wheelchair_accessible`	`Optional[int]`	The wheelchair accessible. Values can be: - 0: No information - 1: Allowed - 2: Not allowed
`bikes_allowed`	`Optional[int]`	The bikes allowed. Values can be: - 0: No information - 1: Allowed - 2: Not allowed
`projects`	`str`	A comma-separated string value for projects that have been applied to this trip.

Source code in network_wrangler/models/gtfs/tables.py

class WranglerTripsTable(TripsTable):
    """Represents the Trips table in the Wrangler feed, adding projects list.

    For field definitions, see the GTFS reference: <https://gtfs.org/documentation/schedule/reference/#tripstxt>

    Attributes:
        trip_id (str): Primary key. Required to be unique.
        shape_id (str): Foreign key to `shape_id` in the shapes table.
        direction_id (DirectionID): The direction id. Required. Values can be:
            - 0: Outbound
            - 1: Inbound
        service_id (str): The service id.
        route_id (str): The route id. Foreign key to `route_id` in the routes table.
        trip_short_name (Optional[str]): The trip short name.
        trip_headsign (Optional[str]): The trip headsign.
        block_id (Optional[str]): The block id.
        wheelchair_accessible (Optional[int]): The wheelchair accessible. Values can be:
            - 0: No information
            - 1: Allowed
            - 2: Not allowed
        bikes_allowed (Optional[int]): The bikes allowed. Values can be:
            - 0: No information
            - 1: Allowed
            - 2: Not allowed
        projects (str): A comma-separated string value for projects that have been applied to this trip.
    """

    projects: Series[str] = pa.Field(coerce=True, default="")

    class Config:
        """Config for the WranglerTripsTable data model."""

        coerce = True
        add_missing_columns = True
        _pk: ClassVar[TablePrimaryKeys] = ["trip_id"]
        _fk: ClassVar[TableForeignKeys] = {"route_id": ("routes", "route_id")}

Stop_times ¶

Wrangler flavor of GTFS StopTimesTable.

For field definitions, see the GTFS reference: https://gtfs.org/documentation/schedule/reference/#stop_timestxt

The primary key of this table is a composite key of trip_id and stop_sequence.

Attributes:

Name	Type	Description
`trip_id`	`str`	Foreign key to `trip_id` in the trips table.
`stop_id`	`int`	Foreign key to `stop_id` in the stops table.
`stop_sequence`	`int`	The stop sequence.
`pickup_type`	`PickupDropoffType`	The pickup type. Values can be: - 0: Regularly scheduled pickup - 1: No pickup available - 2: Must phone agency to arrange pickup - 3: Must coordinate with driver to arrange pickup
`drop_off_type`	`PickupDropoffType`	The drop off type. Values can be: - 0: Regularly scheduled drop off - 1: No drop off available - 2: Must phone agency to arrange drop off - 3: Must coordinate with driver to arrange drop off
`arrival_time`	`datetime`	The arrival time in datetime format.
`departure_time`	`datetime`	The departure time in datetime format.
`shape_dist_traveled`	`Optional[float]`	The shape distance traveled.
`timepoint`	`Optional[TimepointType]`	The timepoint type. Values can be: - 0: The stop is not a timepoint - 1: The stop is a timepoint
`projects`	`str`	A comma-separated string value for projects that have been applied to this stop.

Source code in network_wrangler/models/gtfs/tables.py

class WranglerStopTimesTable(StopTimesTable):
    """Wrangler flavor of GTFS StopTimesTable.

    For field definitions, see the GTFS reference: <https://gtfs.org/documentation/schedule/reference/#stop_timestxt>

    The primary key of this table is a composite key of `trip_id` and `stop_sequence`.

    Attributes:
        trip_id (str): Foreign key to `trip_id` in the trips table.
        stop_id (int): Foreign key to `stop_id` in the stops table.
        stop_sequence (int): The stop sequence.
        pickup_type (PickupDropoffType): The pickup type. Values can be:
            - 0: Regularly scheduled pickup
            - 1: No pickup available
            - 2: Must phone agency to arrange pickup
            - 3: Must coordinate with driver to arrange pickup
        drop_off_type (PickupDropoffType): The drop off type. Values can be:
            - 0: Regularly scheduled drop off
            - 1: No drop off available
            - 2: Must phone agency to arrange drop off
            - 3: Must coordinate with driver to arrange drop off
        arrival_time (datetime.datetime): The arrival time in datetime format.
        departure_time (datetime.datetime): The departure time in datetime format.
        shape_dist_traveled (Optional[float]): The shape distance traveled.
        timepoint (Optional[TimepointType]): The timepoint type. Values can be:
            - 0: The stop is not a timepoint
            - 1: The stop is a timepoint
        projects (str): A comma-separated string value for projects that have been applied to this stop.
    """

    stop_id: Series[int] = pa.Field(nullable=False, coerce=True, description="The model_node_id.")
    arrival_time: Series[Timestamp] = pa.Field(nullable=True, default=pd.NaT, coerce=False)
    departure_time: Series[Timestamp] = pa.Field(nullable=True, default=pd.NaT, coerce=False)
    projects: Series[str] = pa.Field(coerce=True, default="")

    @pa.dataframe_parser
    def parse_times(cls, df):
        """Parse arrival and departure times.

        - Check that all times are timestamps <24h.
        - Check that arrival_time and departure_time are not both "00:00:00".  If so, set
            them to NaT.

        """
        # if arrival_time and departure_time are not set or are both set to "00:00:00", set them to NaT
        if "arrival_time" not in df.columns:
            df["arrival_time"] = pd.NaT
        if "departure_time" not in df.columns:
            df["departure_time"] = pd.NaT
        msg = f"stop_times before parsing: \n {df[['arrival_time', 'departure_time']]}"
        # WranglerLogger.debug(msg)
        filler_timestrings = (df["arrival_time"] == Timestamp("00:00:00")) & (
            df["departure_time"] == Timestamp("00:00:00")
        )

        df.loc[filler_timestrings, "arrival_time"] = pd.NaT
        df.loc[filler_timestrings, "departure_time"] = pd.NaT
        msg = f"stop_times after filling with NaT: \n {df[['arrival_time', 'departure_time']]}"
        # WranglerLogger.debug(msg)
        df["arrival_time"] = str_to_time_series(df["arrival_time"])
        df["departure_time"] = str_to_time_series(df["departure_time"])
        msg = f"stop_times after parsing: \n{df[['arrival_time', 'departure_time']]}"
        # WranglerLogger.debug(msg)
        return df

    class Config:
        """Config for the StopTimesTable data model."""

        coerce = True
        add_missing_columns = True
        _pk: ClassVar[TablePrimaryKeys] = ["trip_id", "stop_sequence"]
        _fk: ClassVar[TableForeignKeys] = {
            "trip_id": ("trips", "trip_id"),
            "stop_id": ("stops", "stop_id"),
        }

        unique: ClassVar[list[str]] = ["trip_id", "stop_sequence"]

Shapes ¶

Wrangler flavor of GTFS ShapesTable.

For field definitions, see the GTFS reference: https://gtfs.org/documentation/schedule/reference/#shapestxt

Attributes:

Name	Type	Description
`shape_id`	`str`	The shape_id. Primary key. Required to be unique.
`shape_pt_lat`	`float`	The shape point latitude.
`shape_pt_lon`	`float`	The shape point longitude.
`shape_pt_sequence`	`int`	The shape point sequence.
`shape_dist_traveled`	`Optional[float]`	The shape distance traveled.
`shape_model_node_id`	`int`	The model_node_id of the shape point. Foreign key to the model_node_id in the nodes table.
`projects`	`str`	A comma-separated string value for projects that have been applied to this shape.

Source code in network_wrangler/models/gtfs/tables.py

class WranglerShapesTable(ShapesTable):
    """Wrangler flavor of GTFS ShapesTable.

     For field definitions, see the GTFS reference: <https://gtfs.org/documentation/schedule/reference/#shapestxt>

    Attributes:
        shape_id (str): The shape_id. Primary key. Required to be unique.
        shape_pt_lat (float): The shape point latitude.
        shape_pt_lon (float): The shape point longitude.
        shape_pt_sequence (int): The shape point sequence.
        shape_dist_traveled (Optional[float]): The shape distance traveled.
        shape_model_node_id (int): The model_node_id of the shape point. Foreign key to the model_node_id in the nodes table.
        projects (str): A comma-separated string value for projects that have been applied to this shape.
    """

    shape_model_node_id: Series[int] = pa.Field(coerce=True, nullable=False)
    projects: Series[str] = pa.Field(coerce=True, default="")

Frequencies ¶

Wrangler flavor of GTFS FrequenciesTable.

For field definitions, see the GTFS reference: https://gtfs.org/documentation/schedule/reference/#frequenciestxt

The primary key of this table is a composite key of trip_id and start_time.

Attributes:

Name	Type	Description
`trip_id`	`str`	Foreign key to `trip_id` in the trips table.
`start_time`	`datetime`	The start time in datetime format.
`end_time`	`datetime`	The end time in datetime format.
`headway_secs`	`int`	The headway in seconds.

Source code in network_wrangler/models/gtfs/tables.py

class WranglerFrequenciesTable(FrequenciesTable):
    """Wrangler flavor of GTFS FrequenciesTable.

    For field definitions, see the GTFS reference: <https://gtfs.org/documentation/schedule/reference/#frequenciestxt>

    The primary key of this table is a composite key of `trip_id` and `start_time`.

    Attributes:
        trip_id (str): Foreign key to `trip_id` in the trips table.
        start_time (datetime.datetime): The start time in datetime format.
        end_time (datetime.datetime): The end time in datetime format.
        headway_secs (int): The headway in seconds.
    """

    projects: Series[str] = pa.Field(coerce=True, default="")
    start_time: Series = pa.Field(
        nullable=False, coerce=True, default=str_to_time(DEFAULT_TIMESPAN[0])
    )
    end_time: Series = pa.Field(
        nullable=False, coerce=True, default=str_to_time(DEFAULT_TIMESPAN[1])
    )

    class Config:
        """Config for the FrequenciesTable data model."""

        coerce = True
        add_missing_columns = True
        unique: ClassVar[list[str]] = ["trip_id", "start_time"]
        _pk: ClassVar[TablePrimaryKeys] = ["trip_id", "start_time"]
        _fk: ClassVar[TableForeignKeys] = {"trip_id": ("trips", "trip_id")}

    @pa.parser("start_time")
    def st_to_timestamp(cls, series: Series) -> Series[Timestamp]:
        """Check that start time is timestamp."""
        series = series.fillna(str_to_time(DEFAULT_TIMESPAN[0]))
        if series.dtype == "datetime64[ns]":
            return series
        series = str_to_time_series(series)
        return series.astype("datetime64[ns]")

    @pa.parser("end_time")
    def et_to_timestamp(cls, series: Series) -> Series[Timestamp]:
        """Check that start time is timestamp."""
        series = series.fillna(str_to_time(DEFAULT_TIMESPAN[1]))
        if series.dtype == "datetime64[ns]":
            return series
        return str_to_time_series(series)

Agencies ¶

Represents the Agency table in the GTFS dataset.

For field definitions, see the GTFS reference: https://gtfs.org/documentation/schedule/reference/#agencytxt

Attributes:

Name	Type	Description
`agency_id`	`str`	The agency_id. Primary key. Required to be unique.
`agency_name`	`str`	The agency name.
`agency_url`	`str`	The agency URL.
`agency_timezone`	`str`	The agency timezone.
`agency_lang`	`str`	The agency language.
`agency_phone`	`str`	The agency phone number.
`agency_fare_url`	`str`	The agency fare URL.
`agency_email`	`str`	The agency email.

Source code in network_wrangler/models/gtfs/tables.py

class AgenciesTable(pa.DataFrameModel):
    """Represents the Agency table in the GTFS dataset.

    For field definitions, see the GTFS reference: <https://gtfs.org/documentation/schedule/reference/#agencytxt>

    Attributes:
        agency_id (str): The agency_id. Primary key. Required to be unique.
        agency_name (str): The agency name.
        agency_url (str): The agency URL.
        agency_timezone (str): The agency timezone.
        agency_lang (str): The agency language.
        agency_phone (str): The agency phone number.
        agency_fare_url (str): The agency fare URL.
        agency_email (str): The agency email.
    """

    agency_id: Series[str] = pa.Field(coerce=True, nullable=False, unique=True)
    agency_name: Series[str] = pa.Field(coerce=True, nullable=True)
    agency_url: Series[HttpURL] = pa.Field(coerce=True, nullable=True)
    agency_timezone: Series[str] = pa.Field(coerce=True, nullable=True)
    agency_lang: Series[str] = pa.Field(coerce=True, nullable=True)
    agency_phone: Series[str] = pa.Field(coerce=True, nullable=True)
    agency_fare_url: Series[str] = pa.Field(coerce=True, nullable=True)
    agency_email: Series[str] = pa.Field(coerce=True, nullable=True)

    class Config:
        """Config for the AgenciesTable data model."""

        coerce = True
        add_missing_columns = True
        _pk: ClassVar[TablePrimaryKeys] = ["agency_id"]

Transit Validation ¶

TransitNetworks can be validated using the following tools:

CLIPython API

python validate_transit.py <network_dir> <file_format> [-s] [--output_dir <output_dir>] [--road_dir <road_dir>] [--road_file_format <road_file_format]

Where:

network_dir: The transit network file directory.
file_format: The suffices of transit network .
--output_dir: The output directory for the validation report.
--road_dir: The directory roadway network. if want to validate the transit network to it.
--road_file_format: The file format for roadway network. Defaults to ‘geojson’.

from network_wrangler.transit.validate import validate_transit_in_dir
validate_transit_in_dir(
    dir=<network_dir>,
    file_format=<network_file_format>,
    road_dir=<road_dir>,
    road_file_format=<road_file_format,
)

Where:

network_dir: The roadway network file directory.
network_file_format: The file format of the transit files.
road_dir: The directory roadway network. if want to validate the transit network to it.
road_file_format: The file format for roadway network. Defaults to ‘geojson’.

Project Cards ¶

Project Cards, which define changes to the roadway and transit networks must use the ProjectCard standard.

Model Roadway Network Export Format ¶

In order to separately calculate the delay when the networks are assigned in static roadway network assignment the roadway network must be exported with separate managed lane links.

To acheive this, RoadwayNetwork objects have the option to be exported in the ModelRoadwayNetwork format, which separates a link into two: set of managed lanes and a set of general purpose lanes which are connected by a set of dummy connector links.

Managed Lane Links ¶

All properties preceded by ML_ will be copied, without that prefix, to the managed lane links.

The following are controlled by parameters which can be set using WranglerConfig:

Geometry of managed lanes will be defined as a shape offset by the parameter ML_OFFSET_METERS. Properties defined in the parameter ADDITIONAL_COPY_FROM_GP_TO_ML are also copied from the parent link.

New model_node_id s and model_link_ids are generated based either on ranges or using a scalar from the GP link based on: ML_LINK_ID_METHOD, ML_NODE_ID_METHOD, ML_LINK_ID_RANGE, ML_NODE_ID_RANGE, ML_LINK_ID_SCALAR, ML_NODE_ID_SCALAR

name is created as “managed lane of name of GP link“

Relationship to the general purpose lanes is retained using the fields GP_A, GP_B, GP_model_link_id.

Managed-lane link-ids are generated as multiples of 10.

Dummy Connector Links ¶

Dummy connector links are generated between the general purpose lane links and managed lane links at points defined by the variable ML_access_point and ML_egress_point. If a managed lane is created without explictly setting these values, network wrangler will assume that the managed lanes can be accessed at any node.

The parameter ADDITIONAL_COPY_TO_ACCESS_EGRESS defines what additional attributes are copied from the general purpose lane to the access and egress links.

name is created as “ dummy link”

model_link_id is created as follows, noting that model_link_id s for managed lanes will be multiples of 10:

1 + managed lane’s model_link_id for access links
2 + managed lane’s model_link_id for access links

Bases: ConfigItem

Model Roadway Configuration.

Attributes:

Name	Type	Description
`ML_OFFSET_METERS`	`int`	Offset in meters for managed lanes.
`ADDITIONAL_COPY_FROM_GP_TO_ML`	`list[str]`	Additional fields to copy from general purpose to managed lanes.
`ADDITIONAL_COPY_TO_ACCESS_EGRESS`	`list[str]`	Additional fields to copy to access and egress links.

Source code in network_wrangler/configs/wrangler.py

@dataclass
class ModelRoadwayConfig(ConfigItem):
    """Model Roadway Configuration.

    Attributes:
        ML_OFFSET_METERS: Offset in meters for managed lanes.
        ADDITIONAL_COPY_FROM_GP_TO_ML: Additional fields to copy from general purpose to managed
            lanes.
        ADDITIONAL_COPY_TO_ACCESS_EGRESS: Additional fields to copy to access and egress links.
    """

    ML_OFFSET_METERS: int = -10
    ADDITIONAL_COPY_FROM_GP_TO_ML: list[str] = Field(default_factory=list)
    ADDITIONAL_COPY_TO_ACCESS_EGRESS: list[str] = Field(default_factory=list)

Bases: ConfigItem

Model Roadway Configuration.

Attributes:

Name	Type	Description
`TRANSIT_SHAPE_ID_METHOD`	`Literal['scalar']`	method for creating a shape_id for a transit shape. Should be “scalar”.
`TRANSIT_SHAPE_ID_SCALAR`	`int`	scalar value to add to general purpose lane to create a shape_id for a transit shape.
`ROAD_SHAPE_ID_METHOD`	`Literal['scalar']`	method for creating a shape_id for a roadway shape. Should be “scalar”.
`ROAD_SHAPE_ID_SCALAR`	`int`	scalar value to add to general purpose lane to create a shape_id for a roadway shape.
`ML_LINK_ID_METHOD`	`Literal['range', 'scalar']`	method for creating a model_link_id for an associated link for a parallel managed lane.
`ML_LINK_ID_RANGE`	`tuple[int, int]`	range of model_link_ids to use when creating an associated link for a parallel managed lane.
`ML_LINK_ID_SCALAR`	`int`	scalar value to add to general purpose lane to create a model_link_id when creating an associated link for a parallel managed lane.
`ML_NODE_ID_METHOD`	`Literal['range', 'scalar']`	method for creating a model_node_id for an associated node for a parallel managed lane.
`ML_NODE_ID_RANGE`	`tuple[int, int]`	range of model_node_ids to use when creating an associated node for a parallel managed lane.
`ML_NODE_ID_SCALAR`	`int`	scalar value to add to general purpose lane node ides create a model_node_id when creating an associated nodes for parallel managed lane.

Source code in network_wrangler/configs/wrangler.py

@dataclass
class IdGenerationConfig(ConfigItem):
    """Model Roadway Configuration.

    Attributes:
        TRANSIT_SHAPE_ID_METHOD: method for creating a shape_id for a transit shape.
            Should be "scalar".
        TRANSIT_SHAPE_ID_SCALAR: scalar value to add to general purpose lane to create a
            shape_id for a transit shape.
        ROAD_SHAPE_ID_METHOD: method for creating a shape_id for a roadway shape.
            Should be "scalar".
        ROAD_SHAPE_ID_SCALAR: scalar value to add to general purpose lane to create a
            shape_id for a roadway shape.
        ML_LINK_ID_METHOD: method for creating a model_link_id for an associated
            link for a parallel managed lane.
        ML_LINK_ID_RANGE: range of model_link_ids to use when creating an associated
            link for a parallel managed lane.
        ML_LINK_ID_SCALAR: scalar value to add to general purpose lane to create a
            model_link_id when creating an associated link for a parallel managed lane.
        ML_NODE_ID_METHOD: method for creating a model_node_id for an associated node
            for a parallel managed lane.
        ML_NODE_ID_RANGE: range of model_node_ids to use when creating an associated
            node for a parallel managed lane.
        ML_NODE_ID_SCALAR: scalar value to add to general purpose lane node ides create
            a model_node_id when creating an associated nodes for parallel managed lane.
    """

    TRANSIT_SHAPE_ID_METHOD: Literal["scalar"] = "scalar"
    TRANSIT_SHAPE_ID_SCALAR: int = 1000000
    ROAD_SHAPE_ID_METHOD: Literal["scalar"] = "scalar"
    ROAD_SHAPE_ID_SCALAR: int = 1000
    ML_LINK_ID_METHOD: Literal["range", "scalar"] = "scalar"
    ML_LINK_ID_RANGE: tuple[int, int] = (950000, 999999)
    ML_LINK_ID_SCALAR: int = 3000000
    ML_NODE_ID_METHOD: Literal["range", "scalar"] = "range"
    ML_NODE_ID_RANGE: tuple[int, int] = (950000, 999999)
    ML_NODE_ID_SCALAR: int = 15000

Network Management ¶

Several functions assist in managing networks themselves including converting serialization formats and clipping to various geographic bounds.

Viewing ¶

Because the roadway network is already in GeoDataFrames, they can be easily viewed in Jupyter Notebooks using commands such as:

my_small_net = SMALL_ROAD.links_df).explore(tiles="CartoDB positron")
SMALL_ROAD.nodes_df.explore(m=my_small_net, color="grey")
my_small_net

For larger networks, you might want to sample objects if you are just trying to get the general picture in order to save memory.

stpaul_net = STPAUL_ROAD.links_df.sample(300).explore(tiles="CartoDB positron")
STPAUL_ROAD.nodes_df.sample(300).explore(m=stpaul_net, color="grey")
stpaul_net

For transit, you have access to GeoDataFrames for both shapes and stops:

STPAUL_TRANSIT.shapes_gdf.explore(m=stpaul_net, color="limegreen")
STPAUL_TRANSIT.stops_gdf.explore(m=stpaul_net, color="limegreen")
stpaul_net

Clipping ¶

There are two options for getting networks that are subsets of the larger one:

clipping networks already loaded from files. Useful when you are already manipulating the objects.
filtering the network when it is read in.

In both instances, you have the option to filter based on one of three methods:

boundary_file which is a geojson or shapefile that you want to filter to.
boundary_gdf passing a geodataframe with a single polygon record that you want to filter to.
boundary_geocode which queries open streetmap for a jurisdiction matching the provided name, e.g. “St Paul, MN, USA”.

Transit additionally has the option to be filtered to a roadway network.

Clipping loaded networks ¶

If your network is already loaded from disk into a RoadwayNetwork or TransitNetwork, you can clip it using clip_roadway() or clip_transit():

from network_wrangler.roadway import clip_roadway
clipped_road_eco = clip_roadway(STPAUL_ROAD, boundary_file=TEST_DATA / "ecolab.geojson")
clipped_road_eco.links_df.explore(m= color="hotpink")

from network_wrangler.transit.clip import clip_transit
clipped_transit_eco = clip_transit(STPAUL_TRANSIT, roadway_net=clipped_road_eco)

Filtering on being read from disk ¶

To filter roadway networks on being read-in, you can use the same parameters (boundary_gdf, boundary_geocode or boundary_file, available in load_roadway_from_dir, and all associated methods.

downtown_net = load_roadway_from_dir(STPAUL_DIR, boundary_geocode="Downtown, St Paul, MN, USA")

This feature is not yet implemented in transit.

Converting Serialization ¶

To convert file serialization without reading it into objects, you can use the method convert_roadway_file_serialization():

from roadway.io import convert_roadway_file_serialization

convert_roadway_file_serialization(
    my_dir, # the path to the input directory.
    "geojson", # the file format of the input files. Defaults to "geojson".
    my_new_dir, # the path were the output will be saved.
    "parquet", # the format of the output files. Defaults to "parquet".
    "new", # the name prefix of the roadway files that will be generated. Defaults to "".
    overwrite = True, # if True, will overwrite the files if they already exist. Defaults to True.
)

clip to

Note you can also pass one of boundary_geocode, boundary_gdf or boundary_file to clip while you are converting the file serialization.