WOLFRAM

Wolfram Language & System Documentation Center
Wolfram Language Home Page »

NetGraph
✖
NetGraph

[Experimental]

✖
NetGraph[{layer1,layer2,…},{m1n1,m2n2,…}]

specifies a neural net defined by a graph in which the output of layer mi is given as input to layer ni.

✖
NetGraph["name1"layer1,"name2"layer2,…,{"namem1""namen1",…}]

specifies a net with explicitly named layers.

✖
NetGraph[layer]

converts a layer or a NetChain into an equivalent minimal NetGraph.

Details and Options

  • NetGraph is typically used to combine neural networks operations with multiple inputs or outputs for operations like residual connections, concatenation, attention and more. The result is a directed acyclic graph.
  • Input or output ports for the entire NetGraph can be created by specifying NetPort["input"]… or …NetPort["output"] in the list of connections.
  • A linear chain of connections within the graph can be specified as layer1layer2…layern, which will cause each layeri to be connected to layeri+1.
  • If the n^(th) layer, or a layer named "layer", has more than one input port or more than one output port, these can be disambiguated by writing NetPort[n,"port"] or NetPort["layer","port"].
  • If a layer has a port that accepts multiple inputs, such as CatenateLayer or ThreadingLayer, multiple connections can be made simultaneously by writing {m1,m2,…}n, which is equivalent to …,m1n,…,m2n,…. The inputs mi are always passed to n in the order m1,m2,….
  • If one or more input or output ports of any layers are left unconnected, these will become ports of the entire NetGraph.
  • If multiple output ports of layers are left unconnected and share the same name "Output", they will become separate ports of the entire NetGraph with names "Output1", "Output2", etc.
  • Some output ports can be left muted by writing NetPort[layer,"port"]None.
  • Specifying the array shapes, a NetEncoder or a NetDecoder for a port can be done with options of the form "port"shape, where shape can be:
  • "Real"a single real number
    "Integer"a single integer
    Restricted["Integer",n]an integer between 1 and n
    Restricted["Integer",{m,n}]an integer between m and n
    na vector of length n
    {n1,n2,…}an array of dimensions n1×n2×…
    "Varying"a vector whose length is variable
    {"Varying",n2,n3,…}an array whose first dimension is variable and remaining dimensions are n2×n3×…
    NetEncoder[…]an encoder (for input ports)
    NetEncoder[{…,"Dimensions"{n1,…}}]an encoder mapped over an array of dimensions n1×…
    NetDecoder[…]a decoder (for output ports)
    NetDecoder[{…,"InputDepth"n}}]a decoder applied to an array of rank n
    FeatureExtractorFunction[…]a feature extractor function
  • NetGraph supports the following special layer specifications in the list of the first argument:
  • Ramp,LogisticSigmoid,…ElementwiseLayer[f]
    Plus,Times,Divide,…ThreadingLayer[f]
    nLinearLayer[n]
    {layer1,layer2,…}NetChain[{layer1,layer2,…}]
  • The following training parameter can be included:
  • LearningRateMultipliersAutomaticlearning rate multipliers for trainable arrays in the net
  • For a net with a single input port, NetGraph[…][data] gives the result of applying the net to data.
  • For a net with multiple input ports, NetGraph[…][<|port1data1,…|>] provides data to each port.
  • For a net with a single output port, NetGraph[…][data] gives the output for that port. If a NetDecoder with a property prop is attached to the output, NetGraph[…][data,prop] computes that property.
  • NetGraph[…][data,"Properties"] can be used to get the list of possible properties.
  • For a net with multiple output ports, NetGraph[…][data] gives an association of the outputs for all ports. NetGraph[…][data,NetPort["oport"]] gives the output for the output port named "oport".
    If a NetDecoder with a property prop is attached to this output, NetGraph[…][data,NetPort["oport"]prop] computes the property for that port.
  • NetGraph[…][data,NetPort["oport"]"Properties"] can be used to get the list of possible properties for a given port.
  • NetGraph[…][data,NetPortGradient["port"]] gives the gradient of the output with respect to the value of the input port "port".
  • NetGraph[…][data,NetPortGradient[{layer1,…,"array"}]] gives the gradient of the output with respect to an array of a nested layer.
  • NetGraph[…][data,NetPortGradient[All]] yields an association with all the gradients.
  • NetGraph[…][data,{spec1,spec2,…}] gives an association of the outputs for all specifications. Each specification speci can be a NetPort setting, a reference to a NetDecoder property or a NetPortGradient.
  • NetGraph[…][data,…,opts] specifies that options should be used in applying the net to data. Possible options include:
  • BatchSizeAutomaticfor lists of inputs, the number of inputs to evaluate at once
    NetEvaluationMode"Test"what mode to use in performing evaluation
    RandomSeedingAutomatichow to seed pseudorandom generators, if any
    TargetDevice"CPU"the target device on which to perform evaluation
    WorkingPrecision"Real32"the numerical precision used for evaluating the net
  • Possible settings for WorkingPrecision include:
  • "Real32"use single-precision real (32-bit)
    "Real64"use double-precision real (64-bit)
    "Mixed"use half-precision real for certain operations
  • WorkingPrecision"Mixed" is only supported for TargetDevice"GPU", where it can result in significant performance increases on certain devices.
  • When given a NumericArray as input, the output will be a NumericArray. In this case, its numeric type is derived from WorkingPrecision.
  • With the setting NetEvaluationMode"Training", layers such as DropoutLayer will behave as they do for training rather than ordinary evaluation.
  • The StandardForm of NetGraph shows the connectivity of layers in the graph and annotates edges with the dimensions of the array that the edge represents. Clicking a layer or a port in the graph shows more information about that layer or port.
  • Normal[NetGraph[…]] returns a list or association of the layers used to construct the graph. EdgeList[NetGraph[…]] returns the list of connections in the graph.
  • NetGraph[…][[spec]] extracts the layer specified by spec from the net.
  • Transformation on NetGraph[…] network can be achieved with NetReplacePart, NetReplace, NetRename, NetFlatten, NetDelete, NetTake, …
  • Options[NetGraph] gives the list of default options to construct the network. Options[NetGraph[…]] gives the list of default options to evaluate the network on some data.
  • Information[NetGraph[…]] gives a report about the network.
  • Information[NetGraph[…],prop] gives the value of property prop of NetGraph[…]. Possible properties include:
  • "Arrays"association giving each array in the network
    "ArraysByteCounts"association giving the byte count of each array
    "ArraysCount"total number of arrays in all layers
    "ArraysDimensions"association giving the dimensions of each array
    "ArraysElementCounts"association giving the number of elements in each array
    "ArraysLearningRateMultipliers"an association of the default learning rate multiplier for each array
    "ArraysPositionList"position specifications of arrays for NetExtract, NetReplacePart, LearningRateMultipliers, …
    "ArraysSizes"association giving the size of each array
    "ArraysTotalByteCount"total number of bytes in all arrays in all layers
    "ArraysTotalElementCount"total number of elements in all arrays
    "ArraysTotalSize"total size of all arrays in all layers
    "FullSummaryGraphic"graphic representing connectivity of all layers in the net, at any depth
    "InputForm"expression to reconstruct the net
    "InputPortNames"list of names of input ports
    "InputPorts"association of input port shapes
    "Layers"association giving each layer in the network
    "LayersCount"total number of layers
    "LayersGraph"graph representing layer connectivity
    "LayersList"list of all layers
    "LayerTypeCounts"how many times each type of layer occurs in the network
    "MXNetNodeGraph"raw graph of underlying "MXNet" operations
    "MXNetNodeGraphPlot"annotated graph of "MXNet" operations
    "OutputPortNames"list of names of output ports
    "OutputPorts"association of output port shapes
    "Properties"available properties
    "RecurrentStatesCount"number of recurrent states in the net
    "RecurrentStatesPositionList"position specifications of recurrent states for NetStateObject
    "SharedArraysCount"total number of shared arrays
    "SummaryGraphic"graphic representing connectivity of layers
  • The properties "ArrayByteCount" and "ArraysTotalByteCount" treat all arrays as if they have already been initialized.

Examples

open allclose all

Basic Examples  (3)Summary of the most common use cases

Create a residual net:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-jlv6v3
Out[1]=1

Initialize all arrays in the net:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-l93cbk
Out[2]=2

Apply the net to an input:

In[3]:=3
✖
https://wolfram.com/xid/0j47f3ve-tfq6w7
Out[3]=3

Convert a NetChain into a NetGraph:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-wdauks
Out[1]=1
In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-za2j2z
Out[2]=2

Nest a layer into a NetGraph:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-yn7g8t
Out[1]=1

Scope  (14)Survey of the scope of standard use cases

Construction  (6)

Construct a net consisting of a linear chain:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-su6evk
Out[1]=1

Some layer names can be omitted and consecutive edge rules can be chained:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-4tp77t
Out[2]=2

The final expressions are identical:

In[3]:=3
✖
https://wolfram.com/xid/0j47f3ve-n49qrr
Out[3]=3

Construct a net graph with an operation requiring two inputs:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-ioczmh
Out[1]=1

Construct a net graph with multiple inputs:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-oybhv2
Out[1]=1

Construct a net graph with multiple outputs:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-xtqh3p
Out[1]=1

Construct a net graph with explicitly named layers:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-yz8rn9
Out[1]=1

Construct a net graph from existing NetGraph or NetChain:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-vjz0mp
Out[1]=1

Special Construction with NetPort  (3)

Create a net where the output is the final internal state of a LongShortTermMemoryLayer:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-hw2rik
Out[1]=1

Apply the net to an input:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-51mluh
Out[2]=2

Modify the connectivity of an existing graph:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-65lh6i
Out[1]=1
In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-6hyfno
Out[2]=2

Connect to the inner port of a wrapped NetChain:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-ndzu2l
Out[1]=1

Flatten the resulting NetGraph:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-8lgj76
Out[2]=2

Evaluation  (3)

Construct a NetGraph:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-rqvyyc
Out[1]=1

Apply the net to an input:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-hoyrct
Out[2]=2

Apply the net to a NumericArray:

In[3]:=3
✖
https://wolfram.com/xid/0j47f3ve-zuhzdg
Out[3]=3

Apply the net using double-precision real:

In[4]:=4
✖
https://wolfram.com/xid/0j47f3ve-r8ffq7
Out[4]=4

Apply the net using the system's default GPU (if any):

In[5]:=5
✖
https://wolfram.com/xid/0j47f3ve-8kgu0z
Out[5]=5

Compute the first-order derivatives of the net:

In[6]:=6
✖
https://wolfram.com/xid/0j47f3ve-gar47i
Out[6]=6

Create a net with a class decoder:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-z166et
Out[1]=1

Evaluate the net:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-1w9n1j
Out[2]=2
In[3]:=3
✖
https://wolfram.com/xid/0j47f3ve-f9h7by
Out[3]=3

Retrieve a property of the decoder by specifying a second argument:

In[4]:=4
✖
https://wolfram.com/xid/0j47f3ve-2z37wy
Out[4]=4
In[5]:=5
✖
https://wolfram.com/xid/0j47f3ve-hyge1h
Out[5]=5
In[6]:=6
✖
https://wolfram.com/xid/0j47f3ve-8rizco
Out[6]=6

Construct a net that explicitly computes a loss:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-5dlgb3
Out[1]=1

Initialize the net and evaluate it on an input:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-u19yh9
In[3]:=3
✖
https://wolfram.com/xid/0j47f3ve-fs6sxh
Out[3]=3

Properties  (2)

Construct a net:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-z4jzdp
Out[1]=1

Extract a given layer by position:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-pty5oi
Out[2]=2

Get the list of layers:

In[3]:=3
✖
https://wolfram.com/xid/0j47f3ve-jkgndr
Out[3]=3

Construct a net with explicitly named layers:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-mtwjrx
Out[1]=1

Extract a given layer by name:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-ss4gtl
Out[2]=2

Get all of the arrays in the net:

In[3]:=3
✖
https://wolfram.com/xid/0j47f3ve-l0edgp
Out[3]=3

Get all layers in an association:

In[4]:=4
✖
https://wolfram.com/xid/0j47f3ve-w2gvad
Out[4]=4

Get the list of edges:

In[5]:=5
✖
https://wolfram.com/xid/0j47f3ve-x9qm49
Out[5]=5

Applications  (1)Sample problems that can be solved with this function

Perform multitask learning by creating a net that produces two separate classifications. First, obtain training data:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-qr7b1u
Out[1]=1

The training data consists of an image and the corresponding high-level and low-level labels:

In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-idfha5
Out[3]=3

Extract the unique labels from the "Label" and "SubLabel" columns:

In[4]:=4
✖
https://wolfram.com/xid/0j47f3ve-ea5daf
Out[4]=4
In[5]:=5
✖
https://wolfram.com/xid/0j47f3ve-zhrhch
Out[5]=5

Create a base convolutional net that will produce a vector of 500 features:

In[7]:=7
✖
https://wolfram.com/xid/0j47f3ve-gd99mo
Out[7]=7

Create a NetGraph that will produce separate classifications for the high-level and low-level labels:

In[8]:=8
✖
https://wolfram.com/xid/0j47f3ve-immcpr
Out[8]=8

Train the network:

In[9]:=9
✖
https://wolfram.com/xid/0j47f3ve-7sabjv

Evaluate the trained network on some example images:

In[10]:=10
✖
https://wolfram.com/xid/0j47f3ve-4nk9wx
Out[10]=10

Get probabilities for a single image:

In[12]:=12
✖
https://wolfram.com/xid/0j47f3ve-nsl5te
Out[12]=12

From a random sample, select the images for which the net produces highest and lowest entropy predictions for "Label":

In[15]:=15
✖
https://wolfram.com/xid/0j47f3ve-re5ehf
In[17]:=17
✖
https://wolfram.com/xid/0j47f3ve-f6okk2
Out[17]=17
Out[18]=18

Use NetTake to produce a sub-network that computes only "SubLabel" predictions:

In[19]:=19
✖
https://wolfram.com/xid/0j47f3ve-evz7o6
Out[19]=19

Make a prediction on a single image:

In[20]:=20
✖
https://wolfram.com/xid/0j47f3ve-67z7t
Out[20]=20

Properties & Relations  (4)Properties of the function, and connections to other functions

NetChain objects can be used as layers in a NetGraph:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-r7owtx
Out[1]=1

NetGraph objects with one input and one output can be used as layers inside NetChain objects:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-cgqknm
Out[1]=1

The layers used to construct a NetGraph can be extracted using Normal:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-q2md4
Out[1]=1
In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-o8pxi1
Out[2]=2

Use Information[graph,"SummaryGraphic"] to get a Graphics[…] expression that shows the underlying connectivity of a graph:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-cufhkh
Out[1]=1
In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-wmr54o
Out[2]=2

Possible Issues  (1)Common pitfalls and unexpected behavior

The order in which edges are defined in a NetGraph can matter. Create a NetGraph that computes a matrix-vector dot product:

In[1]:=1
✖
https://wolfram.com/xid/0j47f3ve-617g11
Out[1]=1
In[2]:=2
✖
https://wolfram.com/xid/0j47f3ve-x1tn72
Out[2]=2

Reversing the order in which the edges are defined causes a failure, as a vector-matrix product with incompatible dimensions is now computed:

In[3]:=3
✖
https://wolfram.com/xid/0j47f3ve-61czia
Out[3]=3
Wolfram Research (2016), NetGraph, Wolfram Language function, https://reference.wolfram.com/language/ref/NetGraph.html (updated 2022).
✖
Wolfram Research (2016), NetGraph, Wolfram Language function, https://reference.wolfram.com/language/ref/NetGraph.html (updated 2022).

Text

Wolfram Research (2016), NetGraph, Wolfram Language function, https://reference.wolfram.com/language/ref/NetGraph.html (updated 2022).

✖
Wolfram Research (2016), NetGraph, Wolfram Language function, https://reference.wolfram.com/language/ref/NetGraph.html (updated 2022).

CMS

Wolfram Language. 2016. "NetGraph." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2022. https://reference.wolfram.com/language/ref/NetGraph.html.

✖
Wolfram Language. 2016. "NetGraph." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2022. https://reference.wolfram.com/language/ref/NetGraph.html.

APA

Wolfram Language. (2016). NetGraph. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/NetGraph.html

✖
Wolfram Language. (2016). NetGraph. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/NetGraph.html

BibTeX

@misc{reference.wolfram_2025_netgraph, author="Wolfram Research", title="{NetGraph}", year="2022", howpublished="\url{https://reference.wolfram.com/language/ref/NetGraph.html}", note=[Accessed: 13-April-2025 ]}

✖
@misc{reference.wolfram_2025_netgraph, author="Wolfram Research", title="{NetGraph}", year="2022", howpublished="\url{https://reference.wolfram.com/language/ref/NetGraph.html}", note=[Accessed: 13-April-2025 ]}

BibLaTeX

@online{reference.wolfram_2025_netgraph, organization={Wolfram Research}, title={NetGraph}, year={2022}, url={https://reference.wolfram.com/language/ref/NetGraph.html}, note=[Accessed: 13-April-2025 ]}

✖
@online{reference.wolfram_2025_netgraph, organization={Wolfram Research}, title={NetGraph}, year={2022}, url={https://reference.wolfram.com/language/ref/NetGraph.html}, note=[Accessed: 13-April-2025 ]}