Skip to content

Design

Atomic Parts

flowchart TD subgraph BaseScenario base_road_net["road_net(RoadwayNetwork)"] base_transit_net["transit_net(TransitNetwork)"] applied_projects["applied_projects(list)"] conflicts["conflicts(dict)"] end BaseScenario --> base_scenario subgraph Scenario base_scenario["base_scenario(dict)"] projects["projects(ProjectCard`"] road_net["road_net(RoadwayNetwork)"] transit_net["transit_net(TransitNetwork)"] config["config(WranglerConfig)"] end

NetworkWrangler deals with four primary atomic parts:

1. Scenario objects describe a Roadway Network, Transit Network, and collection of Projects. Scenarios manage the addition and construction of projects on the network via projct cards including required pre-requisites, co-requisites, and conflicts. Scenarios can be based on or tiered from other scenarios but must at the very least be based on an existing roadway and/or transit network defined in base_scenario.

2. RoadwayNetwork objects stores information about roadway nodes, directed links between nodes, and the shapes of links (note that the same shape can be shared between two or more links). Network Wrangler reads/writes roadway network objects from/to three files: links, shape, and nodes. Their data is stored as GeoDataFrames in the object.

3. TransitNetwork objects contain information about the service (represented by the GTFS-compatable Feed object with stops, routes, trips, shapes, stoptimes, and frequencies tables), and how it maps to a RoadwayNetwork, stored as road_net attribute. Network Wrangler reads/writes transit network information from/to gtfs-like files and stores them as DataFrames within.

4.ProjectCard objects store information (including metadata) about changes to the network. Network Wrangler uses the projectcard package to read project cards from .yaml-like files and validate them.

Applying Projects

The basic functionality of NetworkWrangler is to apply a set of projects to a scenario.

from project_card import read_cards
cards = read_cards([
    "projects/project_a.yml",
    "projects/project_b.yml",
    "projects/project_c.yml",
])
my_scenario.add_project_cards(cards.values())
my_scenario.queued_projects
>>  ["project_a", "project_b", "project_c"]
my_scenario.apply_projects(["project_a"])
my_scenario.applied_projects
>> ["project_a"]
my_scenario.queued_projects
>>  ["project_b", "project_c"]
my_scenario.apply_all_projects()
my_scenario.applied_projects
>> ["project_a", "project_b", "project_c"]

Project Dependencies

Each project can specify any number of other projects (by project name) as a:

  • prerequisite: list of projects that must be applied before this one.
  • corequisite: list of projects that must be applied alongside this one (but not necessarily first).
  • conflicts: list of projects that must not be applied to the same scenario as this one.

The Scenario object keeps track of the applied projects to make sure that these dependencies are enforced.

Base Scenarios with applied projects

Make sure your base scenario contains your applied projects and their conflicts. This can be easily enforced by either loading a scenario from its .yml file or if you are seeding the base_scenario with an actual Scenario object instead of a dictionary.

Order of project application

The order projects are applied defaults to the order they are specified in my_scenario.projects. If a project card lists another project as a prerequisite, then NetworkWrangler will re-arrange the queued projects to make sure that is applied first.

This order can always be reviewed in my_scenario.queued_projects.

Existing Values

You can control the behavior of Network Wrangler if existing value is not equal to the existing attribute in a ProjectCard as follows:

  • error: Raise an error.
  • skip: Skips applying the project.
  • warn: Writes a warning about conflicting values. This is the global default.

These options can be set at the global configuration level by using WranglerConfig.

EDITS:
    EXISTING_VALUE_CONFLICT: warn

This behavior can be overridden within a project card for a roadway_property_change or transit_property_change project for any individual property_changes item.

transit_property_change:
    property_changes:
        headway_secs:
            existing: 360
            change: -180
            existing_value_conflict: error # this overrides the setting in WranglerConfig to error if the existing value is not 360.

Scoped Roadway Property Changes

In some cases, properties of a roadway network may apply only during certain times of the day or for certain categories of users (e.g. trucks, HOV2, etc). The Network Wrangler data model provides the ability to scope each property using the syntax described in the network documentation.

The value for a scoped property for a given link record is a list of ScopedLinkValueItem objects.

Define the value of a link property for a particular timespan or category.

Attributes:

Name Type Description
`category` str

Category or link user that this scoped value applies to, ex: HOV2, truck, etc. Categories are user-defined with the exception of any which is reserved as the default category. Default is DEFAULT_CATEGORY, which is all.

`timespan` list[TimeString]

timespan of the link property as defined as a list of two HH:MM(:SS) strings. Default is DEFAULT_TIMESPAN, which is ["00:00", "24:00"].

`value` Union[float, int, str]

Value of the link property for the given category and timespan.

Conflicting or matching scopes are not allowed in a list of ScopedLinkValueItems:

  • matching: a scope that could be applied for a given category/timespan combination. This includes the default scopes as well as scopes that are contained within the given category AND timespan combination.
  • overlapping: a scope that fully or partially overlaps a given category OR timespan combination. This includes the default scopes, all matching scopes and all scopes where at least one minute of timespan or one category overlap.
  • conflicting: a scope that is overlapping but not matching for a given category/timespan.

NOTE: Default scope values of category: any and timespan:["00:00", "24:00"] are not considered conflicting, but are applied to residual scopes.

There are a few ways that network wrangler can treat conflicts:

  • conflicting (default behavior): overwrite any conflicting scoped properties for that property/link with what is given in the project card.
  • all: overwrite all scoped properties for that property/link with what is given in the project card.
  • error: raise an error when a conflict is detected.

These options can be set at the global configuration level by using WranglerConfig.

EDITS:
    OVERWRITE_SCOPED: error # will default to raising an error if any scoped value in this specific property change conflicts with this scope.

This behavior can be overridden within a project card for a roadway_property_change project for any individual property_changes item.

roadway_property_change:
    property_changes:
        sc_myproperty:
            set: 1
            scoped:
                timespan: [[12:00, 15:00]]
                set: 2
            overwrite_scoped: all # will overwrite all scopes in a link/property. Useful if you are completely defining something rather than amending.

Selections

Roadway

Segment

Example Segment Facility Searches

facility:
    links:
        name: ["6th", "Sixth", "sixth"] # find streets that have one of the various forms of 6th
    from:
        osm_node_id: "187899923"  # start searching for segments at this ID
    to:
        osm_node_id": "187865924" # end at this ID
facility:
    links:
        name: ["6th", "Sixth", "sixth"]
        lanes: [2, 3]  # from the initial connected segment search, only return links that are either 2 OR 3 lanes
    from:
        osm_node_id: "187899923"
    to:
        osm_node_id": "187942339"

When Network Wrangler conducts a search for a facility, it tries to navigate from the from node to the to node based on a network created from the initial selection values of: name, ref, osm_link_id and model_link_id.

If it cannot do so initially, it will expand its search graph several times until it achieves a navigable route – or reaches the maximum number of expansions which defaults to roadway.segment.DEFAULT_MAX_SEARCH_BREADTH = 10 but can be set higher by running ...selection.create_segment(max_search_breadth=BIGGER_NUMBER before returning the values form the selection. In production, you will want to set the search breadth as low as possible while also being successful so that you don’t get strange routes.

Pertinent relationships:

  • RoadwayNetwork.selections is a dictionary of stored RoadwayLinkSelection or RoadwayNodeSelection objects mapped to a hash of the stringified roadway selection dictionary.
  • A RoadwayLinkSelection object which is a segment type of selection will have a single associated Segment object accessed from RoadwayLinkSelection.segment to store relevant functionality.
  • Segment.subnet is a single associated Subnet representing the subset of nodes and links that Network Wrangler will search to find a connected graph. This is because creating a connected graph can be very memory and computationally intensive and we want to limit the size of them substantially.
  • Subnet.graph is the associated networkx.MultiDiGraph connected graph object which is used to conduct the shortest path search.

For an interactive demonstration of what this means: notebooks.Roadway Network Search.ipynb

Organization

../network_wrangler
__init__.py Things that must get done every time network_wrangler is used.
bin Executable scripts.
configs Structure and default values for user-settable configuration.
errors.py User-facing errors.
logger.py Logging utilities and the WranglerLogger class.
models Pydantic and pandera data models and helper functions for them.
params.py Package-wide constants.
roadway Classes and functions pertaining to read, write, analyzing and editing roadway networks.
scenario.py Scenario object class and helper functions.
time.py Time helper functions.
transit Classes and functions pertaining to read, write, analyzing and editing transit networks.
utils Utility functions.
viz.py Visualization helper functions.
network_wrangler/roadway
links Module for managing roadway links.
nodes Module for managing roadway nodes.
projects Module with functions to apply various types of roadway projects.
shapes Module for managing roadway shapes.
clip.py Functions to clip a RoadwayNetwork object to a boundary.
graph.py Functions to convert RoadwayNetwork to osmnx graph and perform graph operations.
io.py Functions for reading and writing roadway networks.
model_roadway.py Functions to create a model roadway network from a roadway network.
network.py Roadway Network class and functions for Network Wrangler.
segment.py Segment class and related functions for working with segments of a RoadwayNetwork.
selection.py Roadway selection classes for selecting links and nodes from a roadway network.
subnet.py Subnet class for RoadwayNetwork object.
utils.py Utility functions for RoadwayNetwork and ModelRoadwayNetwork classes.
validate.py Validates a roadway network to the wrangler data model specifications.
viz.py Visualization functions for RoadwayNetwork and RoadwayLinkSelection.
network_wrangler/transit
feed Relational tables representing transit service.
projects Module with functions to apply various types of transit projects.
clip.py Functions to clip a TransitNetwork object to a boundary.
geo.py Geographic functions for GTFS tables.
io.py Functions for reading and writing transit feeds and networks.
model_transit.py ModelTransit class and functions for managing consistency between roadway and transit networks.
network.py TransitNetwork class for representing a transit network consisting of a schedule Feed mapped to a RoadwayNetwork.
selection.py Classes and functions for selecting transit trips from a transit network.
validate.py Functions to check for transit network validity and consistency with roadway network.