ΦFlow comes with a couple of I/O functions to store fields, such as CenteredGrid
or StaggeredGrid
.
These are stored according to the scene format specification which is designed to store sequences.
from phi.flow import *
A scene stores one sequence of data. Batched data can also be stored in a single scene (see below). To create a new scene, we pass the directory and name. An index suffix is automatically appended to the name.
scene = Scene.create('./data', name='sim')
scene
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/phi/field/_scene.py:150: RuntimeWarning: Failed to copy calling script to scene during Scene.create(): [Errno 2] No such file or directory: '/tmp/ipykernel_3255/2238997793.py' warnings.warn(f"Failed to copy calling script to scene during Scene.create(): {err}", RuntimeWarning)
'./data/sim_000000'
We can list all scenes in the directory.
Scene.list('./data')
('./data/sim_000000',)
Let's write some data to the scene.
Here, we sample random values on a grid according to a smoothness
parameter, which we also want to store.
smoothness = math.random_uniform()
data = CenteredGrid(Noise(smoothness=smoothness + 0.5), x=32, y=16)
plot(data)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/phi/field/_field.py:142: FutureWarning: Instance checks on Grid are deprecated and will be removed in version 3.0. Use the methods instance.is_grid, instance.is_point_cloud, instance.is_centered and instance.is_staggered instead. return isinstance(self, Grid) /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/phi/field/_field.py:148: FutureWarning: Instance checks on PointCloud are deprecated and will be removed in version 3.0. Use the methods instance.is_grid, instance.is_point_cloud, instance.is_centered and instance.is_staggered instead. return isinstance(self, PointCloud)
<Figure size 1200x500 with 2 Axes>
Fields can be stored using the write
method, which creates a new file with the given name and frame index, in this case random_noise_data_000000.npz
.
Additional non-field values, like the smoothness, can be added as properties. Any JSON-serializable object can be stored this way.
scene.write(noise_data=data, frame=0)
scene.put_properties(smoothness=smoothness)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/phi/field/_field_io.py:45: FutureWarning: Instance checks on StaggeredGrid are deprecated and will be removed in version 3.0. Use the methods instance.is_grid, instance.is_point_cloud, instance.is_centered and instance.is_staggered instead. if isinstance(field, StaggeredGrid): /opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/phi/field/_field_io.py:50: FutureWarning: Instance checks on Grid are deprecated and will be removed in version 3.0. Use the methods instance.is_grid, instance.is_point_cloud, instance.is_centered and instance.is_staggered instead. if isinstance(field, Grid):
Next, let's read the data back from disk. We can reference our scene using Scene.list
or Scene.at
.
scene = Scene.at('./data', 0)
The properties of the scene are stored as a dict
in scene.properties
.
scene.properties
{'smoothness': 0.21234188973903656}
Fields can be read using Scene.read
given their name and frame index.
plot(scene.read('noise_data', frame=0))
<Figure size 1200x500 with 2 Axes>
Scenes can also deal with batched data. When batched data is written to a single scene, only one file is created for each stored field.
batched_smoothness = math.random_uniform(batch(batch=4))
batched_data = CenteredGrid(Noise(smoothness=batched_smoothness + 0.5), x=32, y=16)
plot(batched_data, show_color_bar=False)
<Figure size 1200x500 with 4 Axes>
scene.write(batched_noise_data=batched_data, frame=0)
scene.put_properties(smoothness=batched_smoothness)
scene = Scene.at('./data', 0)
plot(scene.read('batched_noise_data'), show_color_bar=False)
<Figure size 1200x500 with 4 Axes>
scene.properties['smoothness']
(0.082, 0.073, 0.933, 0.840) along batchᵇ float64
Scenes themselves can also be batched.
In that case, a single Scene
object represents a batch of actual directories.
When writing data to the batched scene, the data are sliced along the dimensions of the scene.
scene_batch = Scene.create('./data', batch(batch=4), name='sim')
scene_batch
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/phi/field/_scene.py:150: RuntimeWarning: Failed to copy calling script to scene during Scene.create(): [Errno 2] No such file or directory: '/tmp/ipykernel_3255/3852168639.py' warnings.warn(f"Failed to copy calling script to scene during Scene.create(): {err}", RuntimeWarning)
['./data/sim_000001', './data/sim_000002', './data/sim_000003', './data/sim_000004']
scene_batch.write(batched_noise_data=batched_data, frame=0)
scene_batch.put_properties(smoothness=batched_smoothness)
Now, let's look at the contents of a single scene.
scene = Scene.at('./data', 1)
scene.properties['smoothness']
0.08223794400691986
plot(scene.read('batched_noise_data'))
<Figure size 1200x500 with 2 Axes>
If the scene has batch dimensions that are not present on the data, the data are tiled automatically, i.e. stored redundantly across the scenes.