4.5. Loss Recorder
The loss_recorder
module contains classes for recording the loss values during the optimization process.
It is used in the momentum version of the ConFIG algorithm to record the loss values. Not every loss is calculated in a single iteration with the momentum version of the ConFIG algorithm. However, sometimes we need to know the information of all the loss values, e.g., logging and calculating length/weight model. You can create a custom loss recorder by inheriting from the LossRecorder
class.
Loss Recorder¤
conflictfree.loss_recorder.LatestLossRecorder
¤
Bases: LossRecorder
A loss recorder return the latest losses.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
num_losses |
int
|
The number of losses to record |
required |
Source code in conflictfree/loss_recorder.py
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 |
|
record_all_losses
¤
record_all_losses(losses: Sequence) -> list
Records all the losses and returns the recorded losses.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
losses |
Tensor
|
The losses to record. |
required |
Returns:
Name | Type | Description |
---|---|---|
list |
list
|
The recorded losses. |
Source code in conflictfree/loss_recorder.py
33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
|
__init__
¤
__init__(num_losses: int) -> None
Source code in conflictfree/loss_recorder.py
75 76 |
|
record_loss
¤
record_loss(
losses_indexes: Union[int, Sequence[int]],
losses: Union[float, Sequence],
) -> list
Records the given loss and returns the recorded loss.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
losses_indexes |
Union[int, Sequence[int]]
|
The index of the loss. |
required |
losses |
Tensor
|
The loss to record. |
required |
Returns:
Name | Type | Description |
---|---|---|
list |
list
|
The recorded loss. |
Source code in conflictfree/loss_recorder.py
78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 |
|
conflictfree.loss_recorder.MomentumLossRecorder
¤
Bases: LossRecorder
A loss recorder that records the momentum of the loss.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
num_losses |
int
|
The number of losses to record |
required |
betas |
Union[float, Sequence[float]]
|
The moving average constant. |
0.9
|
Source code in conflictfree/loss_recorder.py
97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
|
record_all_losses
¤
record_all_losses(losses: Sequence) -> list
Records all the losses and returns the recorded losses.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
losses |
Tensor
|
The losses to record. |
required |
Returns:
Name | Type | Description |
---|---|---|
list |
list
|
The recorded losses. |
Source code in conflictfree/loss_recorder.py
33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
|
__init__
¤
__init__(
num_losses: int,
betas: Union[float, Sequence[float]] = 0.9,
)
Source code in conflictfree/loss_recorder.py
106 107 108 109 110 111 |
|
record_loss
¤
record_loss(
losses_indexes: Union[int, Sequence[int]],
losses: Union[float, Sequence],
) -> list
Records the given loss and returns the recorded loss.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
losses_indexes |
Union[int, Sequence[int]]
|
The index of the loss. |
required |
losses |
Tensor
|
The loss to record. |
required |
Returns:
Name | Type | Description |
---|---|---|
list |
list
|
The recorded loss. |
Source code in conflictfree/loss_recorder.py
113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
|
Base Class of Loss Recorder¤
conflictfree.loss_recorder.LossRecorder
¤
Base class for loss recorders.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
num_losses |
int
|
The number of losses to record |
required |
Source code in conflictfree/loss_recorder.py
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
|
__init__
¤
__init__(num_losses: int) -> None
Source code in conflictfree/loss_recorder.py
10 11 12 |
|
record_loss
¤
record_loss(
losses_indexes: Union[int, Sequence[int]],
losses: Union[float, Sequence],
) -> list
Records the given loss and returns the recorded losses.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
losses_indexes |
Union[int, Sequence[int]]
|
The index of the loss. |
required |
losses |
Tensor
|
The loss to record. |
required |
Returns:
Name | Type | Description |
---|---|---|
list |
list
|
The recorded losses. |
Raises:
Type | Description |
---|---|
NotImplementedError
|
If the method is not implemented. |
Source code in conflictfree/loss_recorder.py
14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
|
record_all_losses
¤
record_all_losses(losses: Sequence) -> list
Records all the losses and returns the recorded losses.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
losses |
Tensor
|
The losses to record. |
required |
Returns:
Name | Type | Description |
---|---|---|
list |
list
|
The recorded losses. |
Source code in conflictfree/loss_recorder.py
33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
|