Skip to content

simple_neural_fields

SimpleNeuralField(input_size, output_size, potentials_dyn_fcn, input_embedding=None, output_embedding=None, activation_nonlin=torch.sigmoid, tau_init=10.0, tau_learnable=True, kappa_init=0.001, kappa_learnable=True, capacity_learnable=True, potentials_init=None, init_param_kwargs=None, device='cpu')

Bases: PotentialBased

A simplified version of Amari's potential-based recurrent neural network, without the convolution over time.

See Also

[Luksch et al., 2012] T. Luksch, M. Gineger, M. Mühlig, T. Yoshiike, "Adaptive Movement Sequences and Predictive Decisions based on Hierarchical Dynamical Systems", International Conference on Intelligent Robots and Systems, 2012.

output_size: Number of output dimensions. For this simplified neural fields model, the number of outputs
    is equal to the number of neurons in the (single) hidden layer.
input_embedding: Optional (custom) [Module][torch.nn.Module] to extract features from the inputs.
    This module must transform the inputs such that the dimensionality matches the number of
    neurons of the neural field, i.e., `hidden_size`. By default, a [linear layer][torch.nn.Linear]
    without biases is used.
output_embedding: Optional (custom) [Module][torch.nn.Module] to compute the outputs from the activations.
    This module must map the activations of shape (`hidden_size`,) to the outputs of shape (`output_size`,)
    By default, a [linear layer][torch.nn.Linear] without biases is used.
activation_nonlin: Nonlinearity used to compute the activations from the potential levels.
tau_init: Initial value for the shared time constant of the potentials.
tau_learnable: Whether the time constant is a learnable parameter or fixed.
kappa_init: Initial value for the cubic decay, pass 0 to disable the cubic decay.
kappa_learnable: Whether the cubic decay is a learnable parameter or fixed.
capacity_learnable: Whether the capacity is a learnable parameter or fixed.
potentials_init: Initial for the potentials, i.e., the network's hidden state.
init_param_kwargs: Additional keyword arguments for the policy parameter initialization. For example,
    `self_centric_init=True` to initialize the interaction between neurons such that they inhibit the
    others and excite themselves.
device: Device to move this module to (after initialization).
Source code in neuralfields/simple_neural_fields.py
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
def __init__(
    self,
    input_size: int,
    output_size: int,
    potentials_dyn_fcn: PotentialsDynamicsType,
    input_embedding: Optional[nn.Module] = None,
    output_embedding: Optional[nn.Module] = None,
    activation_nonlin: Union[ActivationFunction, Sequence[ActivationFunction]] = torch.sigmoid,
    tau_init: Union[float, int] = 10.0,
    tau_learnable: bool = True,
    kappa_init: Union[float, int] = 1e-3,
    kappa_learnable: bool = True,
    capacity_learnable: bool = True,
    potentials_init: Optional[torch.Tensor] = None,
    init_param_kwargs: Optional[dict] = None,
    device: Union[str, torch.device] = "cpu",
):
    """
    Args:
        input_size: Number of input dimensions.
        output_size: Number of output dimensions. For this simplified neural fields model, the number of outputs
            is equal to the number of neurons in the (single) hidden layer.
        input_embedding: Optional (custom) [Module][torch.nn.Module] to extract features from the inputs.
            This module must transform the inputs such that the dimensionality matches the number of
            neurons of the neural field, i.e., `hidden_size`. By default, a [linear layer][torch.nn.Linear]
            without biases is used.
        output_embedding: Optional (custom) [Module][torch.nn.Module] to compute the outputs from the activations.
            This module must map the activations of shape (`hidden_size`,) to the outputs of shape (`output_size`,)
            By default, a [linear layer][torch.nn.Linear] without biases is used.
        activation_nonlin: Nonlinearity used to compute the activations from the potential levels.
        tau_init: Initial value for the shared time constant of the potentials.
        tau_learnable: Whether the time constant is a learnable parameter or fixed.
        kappa_init: Initial value for the cubic decay, pass 0 to disable the cubic decay.
        kappa_learnable: Whether the cubic decay is a learnable parameter or fixed.
        capacity_learnable: Whether the capacity is a learnable parameter or fixed.
        potentials_init: Initial for the potentials, i.e., the network's hidden state.
        init_param_kwargs: Additional keyword arguments for the policy parameter initialization. For example,
            `self_centric_init=True` to initialize the interaction between neurons such that they inhibit the
            others and excite themselves.
        device: Device to move this module to (after initialization).
    """
    init_param_kwargs = init_param_kwargs if init_param_kwargs is not None else dict()

    # Create the common layers and parameters.
    super().__init__(
        input_size=input_size,
        hidden_size=output_size,
        output_size=output_size,
        activation_nonlin=activation_nonlin,
        tau_init=tau_init,
        tau_learnable=tau_learnable,
        kappa_init=kappa_init,
        kappa_learnable=kappa_learnable,
        potentials_init=potentials_init,
        input_embedding=input_embedding,
        output_embedding=output_embedding,
        device=device,
    )

    # Create the layer that converts the activations of the previous time step into potentials (internal stimulus).
    # For this model, self._hidden_size equals output_size.
    self.prev_activations_embedding = nn.Linear(self._hidden_size, self._hidden_size, bias=False)
    init_param_(self.prev_activations_embedding, **init_param_kwargs)

    # Create the layer that converts potentials into activations which are the outputs in this model.
    # Scaling weights equals beta in eq (4) in [Luksch et al., 2012].
    self.potentials_to_activations = IndependentNonlinearitiesLayer(
        self._hidden_size, nonlin=activation_nonlin, bias=False, weight=True
    )

    # Potential dynamics' capacity.
    self.potentials_dyn_fcn = potentials_dyn_fcn
    self.capacity_learnable = capacity_learnable
    if self.potentials_dyn_fcn in [pd_capacity_21, pd_capacity_21_abs, pd_capacity_32, pd_capacity_32_abs]:
        if _is_iterable(activation_nonlin):
            self._capacity_opt_init = self._init_capacity_heuristic(activation_nonlin[0])
        else:
            self._capacity_opt_init = self._init_capacity_heuristic(activation_nonlin)  # type: ignore[arg-type]
    else:
        # Even if the potential function does not include a capacity term, we initialize it to be compatible with
        # custom functions.
        self._capacity_opt_init = torch.tensor(1.0, dtype=torch.get_default_dtype())
    self._capacity_opt = nn.Parameter(self._capacity_opt_init.to(device=device), requires_grad=capacity_learnable)

    # Move the complete model to the given device.
    self.to(device=device)

capacity: Union[torch.Tensor, nn.Parameter] property

Get the capacity parameter (only used for capacity-based dynamics functions).

device: torch.device property

Get the device this model is located on. This assumes that all parts are located on the same device.

hidden_size: int property

Get the number of neurons in the neural field layer, i.e., the ones with the in-/exhibition dynamics.

kappa: Union[torch.Tensor, nn.Parameter] property

Get the cubic decay parameter \(\kappa\).

param_values: torch.Tensor property writable

Get the module's parameters as a 1-dimensional array. The values are copied, thus modifying the return value does not propagate back to the module parameters.

stimuli_external: torch.Tensor property

Get the neurons' external stimuli, resulting from the current inputs. This property is useful for recording during a simulation / rollout.

stimuli_internal: torch.Tensor property

Get the neurons' internal stimuli, resulting from the previous activations of the neurons. This property is useful for recording during a simulation / rollout.

tau: Union[torch.Tensor, nn.Parameter] property

Get the timescale parameter, called \(\tau\) in the original paper [Amari_77].

transform_to_img_space: Callable[[torch.Tensor], torch.Tensor] = torch.exp class-attribute instance-attribute

Function to map parameters to the image space of the original problem.

transform_to_opt_space: Callable[[torch.Tensor], torch.Tensor] = torch.log class-attribute instance-attribute

Function to map parameters to the optimization space.

forward(inputs, hidden=None)

Compute the external and internal stimuli, advance the potential dynamics for one time step, and return the model's output for several time steps in a row.

This method essentially calls forward_one_step several times in a row.

Parameters:

Name Type Description Default
inputs Tensor

Inputs of shape (batch_size, num_steps, dim_input) to evaluate the network on.

required
hidden Optional[Tensor]

Initial values of the hidden states, i.e., the potentials. By default, the network initialized the hidden state to be all zeros. However, via this argument one can set a specific initial value for the potentials. Depending on the shape of inputs, hidden is of shape (hidden_size,) if the input was not batched, else of shape (batch_size, hidden_size).

None

Returns:

Type Description
Tensor

The outputs, i.e., the (linearly combined) activations, and all intermediate potential values, both of

Tensor

shape (batch_size, num_steps, dim_output).

Source code in neuralfields/potential_based.py
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
def forward(self, inputs: torch.Tensor, hidden: Optional[torch.Tensor] = None) -> Tuple[torch.Tensor, torch.Tensor]:
    """Compute the external and internal stimuli, advance the potential dynamics for one time step, and return
    the model's output for several time steps in a row.

    This method essentially calls [forward_one_step][neuralfields.PotentialBased.forward_one_step] several times
    in a row.

    Args:
        inputs: Inputs of shape `(batch_size, num_steps, dim_input)` to evaluate the network on.
        hidden: Initial values of the hidden states, i.e., the potentials. By default, the network initialized
            the hidden state to be all zeros. However, via this argument one can set a specific initial value
            for the potentials. Depending on the shape of `inputs`, `hidden` is of shape `(hidden_size,)` if
            the input was not batched, else of shape `(batch_size, hidden_size)`.

    Returns:
        The outputs, i.e., the (linearly combined) activations, and all intermediate potential values, both of
        shape `(batch_size, num_steps, dim_output)`.
    """
    # Bring the sequence of inputs into the shape (batch_size, num_steps, dim_input).
    batch_size = PotentialBased._infer_batch_size(inputs)
    inputs = inputs.view(batch_size, -1, self.input_size)  # moved to the desired device by forward_one_step() later

    # If given use the hidden tensor, i.e., the potentials of the last step, else initialize them.
    hidden = self.init_hidden(batch_size, hidden)  # moved to the desired device by forward_one_step() later

    # Iterate over the time dimension. Do this in parallel for all batched which are still along the 1st dimension.
    inputs = inputs.permute(1, 0, 2)  # move time to first dimension for easy iterating
    outputs_all = []
    hidden_all = []
    for inp in inputs:
        outputs, hidden_next = self.forward_one_step(inp, hidden)
        hidden = hidden_next.clone()
        outputs_all.append(outputs)
        hidden_all.append(hidden_next)

    # Return the outputs and hidden states, both stacked along the time dimension.
    return torch.stack(outputs_all, dim=1), torch.stack(hidden_all, dim=1)

init_hidden(batch_size=None, potentials_init=None)

Provide initial values for the hidden parameters. This usually is a zero tensor.

Parameters:

Name Type Description Default
batch_size Optional[int]

Number of batches, i.e., states to track in parallel.

None
potentials_init Optional[Tensor]

Initial values for the potentials to override the networks default values with.

None

Returns:

Type Description
Union[Tensor, Parameter]

Tensor of shape (hidden_size,) if hidden was not batched, else of shape (batch_size, hidden_size).

Source code in neuralfields/potential_based.py
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
def init_hidden(
    self, batch_size: Optional[int] = None, potentials_init: Optional[torch.Tensor] = None
) -> Union[torch.Tensor, torch.nn.Parameter]:
    """Provide initial values for the hidden parameters. This usually is a zero tensor.

    Args:
        batch_size: Number of batches, i.e., states to track in parallel.
        potentials_init: Initial values for the potentials to override the networks default values with.

    Returns:
        Tensor of shape `(hidden_size,)` if `hidden` was not batched, else of shape `(batch_size, hidden_size)`.
    """
    if potentials_init is None:
        if batch_size is None:
            return self._potentials_init.view(-1)
        return self._potentials_init.repeat(batch_size, 1).to(device=self.device)

    return potentials_init.to(device=self.device)

potentials_dot(potentials, stimuli)

Compute the derivative of the neurons' potentials per time step.

\(/tau /dot{u} = f(u, s, h)\) with the potentials \(u\), the combined stimuli \(s\), and the resting level \(h\).

Parameters:

Name Type Description Default
potentials Tensor

Potential values at the current point in time, of shape (hidden_size,).

required
stimuli Tensor

Sum of external and internal stimuli at the current point in time, of shape (hidden_size,).

required

Returns:

Type Description
Tensor

Time derivative of the potentials \(\frac{dp}{dt}\), of shape (hidden_size,).

Source code in neuralfields/simple_neural_fields.py
389
390
391
392
393
394
395
396
397
398
399
400
401
402
def potentials_dot(self, potentials: torch.Tensor, stimuli: torch.Tensor) -> torch.Tensor:
    r"""Compute the derivative of the neurons' potentials per time step.

    $/tau /dot{u} = f(u, s, h)$
    with the potentials $u$, the combined stimuli $s$, and the resting level $h$.

    Args:
        potentials: Potential values at the current point in time, of shape `(hidden_size,)`.
        stimuli: Sum of external and internal stimuli at the current point in time, of shape `(hidden_size,)`.

    Returns:
        Time derivative of the potentials $\frac{dp}{dt}$, of shape `(hidden_size,)`.
    """
    return self.potentials_dyn_fcn(potentials, stimuli, self.resting_level, self.tau, self.kappa, self.capacity)

pd_capacity_21(p, s, h, tau, kappa, capacity)

Capacity-based dynamics with 2 stable (\(p=-C\), \(p=C\)) and 1 unstable fix points (\(p=0\)) for \(s=0\)

\(\tau \dot{p} = s - (h - p) (1 - \frac{(h - p)^2}{C^2})\)

Notes

This potential dynamics function is strongly recommended to be used with a tanh activation function.

Parameters:

Name Type Description Default
p Tensor

Potential, higher values lead to higher activations.

required
s Tensor

Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).

required
h Tensor

Resting level, a.k.a. constant offset.

required
tau Tensor

Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).

required
kappa Optional[Tensor]

Cubic decay factor for a neuron's potential, ignored for this dynamics function.

required
capacity Optional[Tensor]

Capacity value of a neuron's potential.

required

Returns:

Type Description
Tensor

Time derivative of the potentials \(\frac{dp}{dt}\).

Source code in neuralfields/simple_neural_fields.py
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
def pd_capacity_21(
    p: torch.Tensor,
    s: torch.Tensor,
    h: torch.Tensor,
    tau: torch.Tensor,
    kappa: Optional[torch.Tensor],
    capacity: Optional[torch.Tensor],
) -> torch.Tensor:
    r"""Capacity-based dynamics with 2 stable ($p=-C$, $p=C$) and 1 unstable fix points ($p=0$) for $s=0$

    $\tau \dot{p} =  s - (h - p) (1 - \frac{(h - p)^2}{C^2})$

    Notes:
        This potential dynamics function is strongly recommended to be used with a [tanh][torch.tanh] activation
        function.

    Args:
        p: Potential, higher values lead to higher activations.
        s: Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).
        h: Resting level, a.k.a. constant offset.
        tau: Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).
        kappa: Cubic decay factor for a neuron's potential, ignored for this dynamics function.
        capacity: Capacity value of a neuron's potential.

    Returns:
        Time derivative of the potentials $\frac{dp}{dt}$.
    """
    _verify_tau(tau)
    _verify_capacity(capacity)
    return (s - (h - p) * (torch.ones_like(p) - (h - p) ** 2 / capacity**2)) / tau

pd_capacity_21_abs(p, s, h, tau, kappa, capacity)

Capacity-based dynamics with 2 stable (\(p=-C\), \(p=C\)) and 1 unstable fix points (\(p=0\)) for \(s=0\)

\(\tau \dot{p} = s - (h - p) (1 - \frac{\left| h - p \right|}{C})\)

The "absolute version" of pd_capacity_21 has a lower magnitude and a lower oder of the resulting polynomial.

Notes

This potential dynamics function is strongly recommended to be used with a tanh activation function.

Parameters:

Name Type Description Default
p Tensor

Potential, higher values lead to higher activations.

required
s Tensor

Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).

required
h Tensor

Resting level, a.k.a. constant offset.

required
tau Tensor

Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).

required
kappa Optional[Tensor]

Cubic decay factor for a neuron's potential, ignored for this dynamics function.

required
capacity Optional[Tensor]

Capacity value of a neuron's potential.

required

Returns:

Type Description
Tensor

Time derivative of the potentials \(\frac{dp}{dt}\).

Source code in neuralfields/simple_neural_fields.py
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
def pd_capacity_21_abs(
    p: torch.Tensor,
    s: torch.Tensor,
    h: torch.Tensor,
    tau: torch.Tensor,
    kappa: Optional[torch.Tensor],
    capacity: Optional[torch.Tensor],
) -> torch.Tensor:
    r"""Capacity-based dynamics with 2 stable ($p=-C$, $p=C$) and 1 unstable fix points ($p=0$) for $s=0$

    $\tau \dot{p} =  s - (h - p) (1 - \frac{\left| h - p \right|}{C})$

    The "absolute version" of `pd_capacity_21` has a lower magnitude and a lower oder of the resulting polynomial.

    Notes:
        This potential dynamics function is strongly recommended to be used with a [tanh][torch.tanh] activation
        function.

    Args:
        p: Potential, higher values lead to higher activations.
        s: Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).
        h: Resting level, a.k.a. constant offset.
        tau: Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).
        kappa: Cubic decay factor for a neuron's potential, ignored for this dynamics function.
        capacity: Capacity value of a neuron's potential.

    Returns:
        Time derivative of the potentials $\frac{dp}{dt}$.
    """
    _verify_tau(tau)
    _verify_capacity(capacity)
    return (s - (h - p) * (torch.ones_like(p) - torch.abs(h - p) / capacity)) / tau

pd_capacity_32(p, s, h, tau, kappa, capacity)

Capacity-based dynamics with 3 stable (\(p=-C\), \(p=0\), \(p=C\)) and 2 unstable fix points (\(p=-C/2\), \(p=C/2\)) for \(s=0\)

\(\tau \dot{p} = s - (h - p) (1 - \frac{(h - p)^2}{C^2}) (1 - \frac{(2(h - p))^2}{C^2})\)

Notes

This potential dynamics function is strongly recommended to be used with a tanh activation function.

Parameters:

Name Type Description Default
p Tensor

Potential, higher values lead to higher activations.

required
s Tensor

Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).

required
h Tensor

Resting level, a.k.a. constant offset.

required
tau Tensor

Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).

required
kappa Optional[Tensor]

Cubic decay factor for a neuron's potential, ignored for this dynamics function.

required
capacity Optional[Tensor]

Capacity value of a neuron's potential.

required

Returns:

Type Description
Tensor

Time derivative of the potentials \(\frac{dp}{dt}\).

Source code in neuralfields/simple_neural_fields.py
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
def pd_capacity_32(
    p: torch.Tensor,
    s: torch.Tensor,
    h: torch.Tensor,
    tau: torch.Tensor,
    kappa: Optional[torch.Tensor],
    capacity: Optional[torch.Tensor],
) -> torch.Tensor:
    r"""Capacity-based dynamics with 3 stable ($p=-C$, $p=0$, $p=C$) and 2 unstable fix points ($p=-C/2$, $p=C/2$)
    for $s=0$

    $\tau \dot{p} =  s - (h - p) (1 - \frac{(h - p)^2}{C^2}) (1 - \frac{(2(h - p))^2}{C^2})$

    Notes:
        This potential dynamics function is strongly recommended to be used with a [tanh][torch.tanh] activation
        function.

    Args:
        p: Potential, higher values lead to higher activations.
        s: Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).
        h: Resting level, a.k.a. constant offset.
        tau: Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).
        kappa: Cubic decay factor for a neuron's potential, ignored for this dynamics function.
        capacity: Capacity value of a neuron's potential.

    Returns:
        Time derivative of the potentials $\frac{dp}{dt}$.
    """
    _verify_tau(tau)
    _verify_capacity(capacity)
    return (
        s
        + (h - p)
        * (torch.ones_like(p) - (h - p) ** 2 / capacity**2)
        * (torch.ones_like(p) - ((2 * (h - p)) ** 2 / capacity**2))
    ) / tau

pd_capacity_32_abs(p, s, h, tau, kappa, capacity)

Capacity-based dynamics with 3 stable (\(p=-C\), \(p=0\), \(p=C\)) and 2 unstable fix points (\(p=-C/2\), \(p=C/2\)) for \(s=0\).

\(\tau \dot{p} = \left( s + (h - p) (1 - \frac{\left| (h - p) \right|}{C}) (1 - \frac{2 \left| (h - p) \right|}{C}) \right)\)

The "absolute version" of pd_capacity_32 is less skewed due to a lower oder of the resulting polynomial.

Notes

This potential dynamics function is strongly recommended to be used with a tanh activation function.

Parameters:

Name Type Description Default
p Tensor

Potential, higher values lead to higher activations.

required
s Tensor

Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).

required
h Tensor

Resting level, a.k.a. constant offset.

required
tau Tensor

Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).

required
kappa Optional[Tensor]

Cubic decay factor for a neuron's potential, ignored for this dynamics function.

required
capacity Optional[Tensor]

Capacity value of a neuron's potential.

required

Returns:

Type Description
Tensor

Time derivative of the potentials \(\frac{dp}{dt}\).

Source code in neuralfields/simple_neural_fields.py
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
def pd_capacity_32_abs(
    p: torch.Tensor,
    s: torch.Tensor,
    h: torch.Tensor,
    tau: torch.Tensor,
    kappa: Optional[torch.Tensor],
    capacity: Optional[torch.Tensor],
) -> torch.Tensor:
    r"""Capacity-based dynamics with 3 stable ($p=-C$, $p=0$, $p=C$) and 2 unstable fix points ($p=-C/2$, $p=C/2$)
    for $s=0$.

    $\tau \dot{p} =  \left( s + (h - p) (1 - \frac{\left| (h - p) \right|}{C})
    (1 - \frac{2 \left| (h - p) \right|}{C}) \right)$

    The "absolute version" of `pd_capacity_32` is less skewed due to a lower oder of the resulting polynomial.

    Notes:
        This potential dynamics function is strongly recommended to be used with a [tanh][torch.tanh] activation
        function.

    Args:
        p: Potential, higher values lead to higher activations.
        s: Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).
        h: Resting level, a.k.a. constant offset.
        tau: Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).
        kappa: Cubic decay factor for a neuron's potential, ignored for this dynamics function.
        capacity: Capacity value of a neuron's potential.

    Returns:
        Time derivative of the potentials $\frac{dp}{dt}$.
    """
    _verify_tau(tau)
    _verify_capacity(capacity)
    return (
        s
        + (h - p)
        * (torch.ones_like(p) - torch.abs(h - p) / capacity)
        * (torch.ones_like(p) - 2 * torch.abs(h - p) / capacity)
    ) / tau

pd_cubic(p, s, h, tau, kappa, capacity)

Basic proportional dynamics with additional cubic decay.

\(\tau \dot{p} = s + h - p + \kappa (h - p)^3\)

Notes

This potential dynamics function is strongly recommended to be used with a sigmoid activation function.

Parameters:

Name Type Description Default
p Tensor

Potential, higher values lead to higher activations.

required
s Tensor

Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).

required
h Tensor

Resting level, a.k.a. constant offset.

required
tau Tensor

Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).

required
kappa Optional[Tensor]

Cubic decay factor for a neuron's potential.

required
capacity Optional[Tensor]

Capacity value of a neuron's potential, ignored for this dynamics function.

required

Returns:

Type Description
Tensor

Time derivative of the potentials \(\frac{dp}{dt}\).

Source code in neuralfields/simple_neural_fields.py
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
def pd_cubic(
    p: torch.Tensor,
    s: torch.Tensor,
    h: torch.Tensor,
    tau: torch.Tensor,
    kappa: Optional[torch.Tensor],
    capacity: Optional[torch.Tensor],
) -> torch.Tensor:
    r"""Basic proportional dynamics with additional cubic decay.

    $\tau \dot{p} = s + h - p + \kappa (h - p)^3$

    Notes:
        This potential dynamics function is strongly recommended to be used with a [sigmoid][torch.sigmoid] activation
        function.

    Args:
        p: Potential, higher values lead to higher activations.
        s: Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).
        h: Resting level, a.k.a. constant offset.
        tau: Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).
        kappa: Cubic decay factor for a neuron's potential.
        capacity: Capacity value of a neuron's potential, ignored for this dynamics function.

    Returns:
        Time derivative of the potentials $\frac{dp}{dt}$.
    """
    _verify_tau(tau)
    _verify_kappa(kappa)
    return (s + h - p + kappa * torch.pow(h - p, 3)) / tau

pd_linear(p, s, h, tau, kappa, capacity)

Basic proportional dynamics.

\(\tau \dot{p} = s - p\)

Notes

This potential dynamics function is strongly recommended to be used with a sigmoid activation function.

Parameters:

Name Type Description Default
p Tensor

Potential, higher values lead to higher activations.

required
s Tensor

Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).

required
h Tensor

Resting level, a.k.a. constant offset.

required
tau Tensor

Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).

required
kappa Optional[Tensor]

Cubic decay factor for a neuron's potential, ignored for this dynamics function.

required
capacity Optional[Tensor]

Capacity value of a neuron's potential, ignored for this dynamics function.

required

Returns:

Type Description
Tensor

Time derivative of the potentials \(\frac{dp}{dt}\).

Source code in neuralfields/simple_neural_fields.py
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
def pd_linear(
    p: torch.Tensor,
    s: torch.Tensor,
    h: torch.Tensor,
    tau: torch.Tensor,
    kappa: Optional[torch.Tensor],
    capacity: Optional[torch.Tensor],
) -> torch.Tensor:
    r"""Basic proportional dynamics.

    $\tau \dot{p} = s - p$

    Notes:
        This potential dynamics function is strongly recommended to be used with a [sigmoid][torch.sigmoid] activation
        function.

    Args:
        p: Potential, higher values lead to higher activations.
        s: Stimulus, higher values lead to larger changes of the potentials (depends on the dynamics function).
        h: Resting level, a.k.a. constant offset.
        tau: Time scaling factor, higher values lead to slower changes of the potentials (linear dependency).
        kappa: Cubic decay factor for a neuron's potential, ignored for this dynamics function.
        capacity: Capacity value of a neuron's potential, ignored for this dynamics function.

    Returns:
        Time derivative of the potentials $\frac{dp}{dt}$.
    """
    _verify_tau(tau)
    return (s + h - p) / tau