GitHub

Saving and loading files

Introduction

For some of our datatypes we provide a way to save them in and load them from JSON format. The most common OSCAR types are supported, but it will take some time until all corners of OSCAR are covered by this effort. Our overall goal is threefold:

  • Avoid recomputation by providing an easy way to store data.
  • Increase portability by giving a convenient possibility to transport data.
  • Increase overall software quality by testing against existing data and tracking errors through data computed by different versions of OSCAR (or other computer algebra systems).

For more details read the developer documentation. Work on serialization is supported by the MaRDI project. You can find out more about its Task Area 1 (Computer Algebra) here.

save — Function
save(io::IO, obj::Any; metadata::MetaData=nothing, with_attrs::Bool=true, serializer::OscarSerializer=JSONSerializer())
save(filename::String, obj::Any; metadata::MetaData=nothing, with_attrs::Bool=true, compression::Symbol=:none, pretty_print::Bool=false, serializer::OscarSerializer=JSONSerializer())

Save an object obj to the given io stream respectively to the file filename. When used with with_attrs=true the object will save its attributes along with all the attributes of the types used in the object's struct. The attributes that will be saved are defined during type registration, see @register_serialization_type.

The optional serializer argument controls the output format and layout. The default JSONSerializer writes a single .mrdi file. Other serializers such as MultiFileRefSerializer (multi-file prefix-based) and LPSerializer (external LP file for linear programs) are available. See the serialization documentation for details and examples.

Setting the optional argument compression will compress the file using the given compression method. The filename must have the appropriate file extension for the chosen compression method. Currently, only :none (default) and :gzip are supported.

The pretty_print optional argument can be used similar to the standard JSON functionality.

See load.

Examples

julia> meta = metadata(author_orcid="0000-0000-0000-0042", name="42", description="The meaning of life, the universe and everything")
Oscar.Serialization.MetaData("0000-0000-0000-0042", "42", "The meaning of life, the universe and everything")

julia> save("fourtitwo.mrdi", 42; metadata=meta);

julia> read_metadata("fourtitwo.mrdi")
{
  "author_orcid": "0000-0000-0000-0042",
  "name": "42",
  "description": "The meaning of life, the universe and everything"
}

julia> load("fourtitwo.mrdi")
42
source
load — Function
load(io::IO; params::Any = nothing, type::Any = nothing, with_attrs::Bool=true, serializer::OscarSerializer=JSONSerializer())
load(filename::String; params::Any = nothing, type::Any = nothing, with_attrs::Bool=true, serializer::OscarSerializer=JSONSerializer())

Load the object stored in the given io stream respectively in the file filename.

If params is specified, then the root object of the loaded data will attempt a load using these parameters. In the case of rings this results in setting its parent, or in the case of a container of ring types such as Vector or Tuple, the parent of the entries will be set using their params.

If a type T is given then attempt to load the root object of the data being loaded with this type; if this fails, an error is thrown.

If with_attrs=true the object will be loaded with attributes available from the file (or serialized data).

The optional serializer argument must match the one used when saving. Pass the same serializer instance (e.g. MultiFileRefSerializer() or LPSerializer(basepath)) that was used with save. See the serialization documentation for details and examples.

If the file was created with compression=:gzip and the filename ends in .gz, the file will be decompressed on-the-fly automatically.

See save.

Examples

julia> save("fourtitwo.mrdi", 42);

julia> load("fourtitwo.mrdi")
42

julia> load("fourtitwo.mrdi"; type=Int64)
42

julia> R, x = QQ[:x]
(Univariate polynomial ring in x over QQ, x)

julia> p = x^2 - x + 1
x^2 - x + 1

julia> save("p.mrdi", p)

julia> p_loaded = load("p.mrdi", params=R)
x^2 - x + 1

julia> parent(p_loaded) === R
true

julia> save("p_v.mrdi", [p, p])

julia> loaded_p_v = load("p_v.mrdi", params=R)
2-element Vector{QQPolyRingElem}:
 x^2 - x + 1
 x^2 - x + 1

julia> parent(loaded_p_v[1]) === parent(loaded_p_v[2]) === R
true
source

Serializers

OSCAR provides several serializers for different use cases. The serializer is passed via the serializer keyword argument to save and load.

JSONSerializer (default)

The default serializer. Writes a single .mrdi JSON file. Referenced objects (e.g. the parent ring of a polynomial) are stored inline under the _refs key.

julia> R, x = QQ[:x];

julia> p = x^2 + 1
x^2 + 1

julia> save("poly.mrdi", p)

julia> save("poly.mrdi", p; serializer=Oscar.Serialization.JSONSerializer(serialize_refs=false))

MultiFileRefSerializer

Splits referenced objects into separate files alongside the main file, using the path as a prefix. Each referenced object is written to its own <prefix>_<UUID>.mrdi file. The main file is <prefix>.mrdi. Use this when objects share large sub-objects.

The filename argument is used as a prefix directly — no extension is added or stripped. On overwrite, stale ref files from the previous save are removed automatically.

Without compression:

julia> Qx, x = QQ[:x];

julia> F, a = number_field(x^2 + 2);

julia> R, (y, z) = F[:y, :z];

julia> p = a * y - z;

julia> poly_dir = mkdir("poly_dir");

julia> save(joinpath(poly_dir, "polydata"), p; serializer=Oscar.Serialization.MultiFileRefSerializer())

julia> readdir(poly_dir)
4-element Vector{String}:
 "polydata.mrdi"
 "polydata_52e4c5e2-ff94-4b1c-9832-a61d8a54331a.mrdi"
 "polydata_c8be128f-a28c-4d08-a67f-5e1c2ad9409d.mrdi"
 "polydata_f2bd8b4b-e6a7-4961-943b-1d68306889a2.mrdi"
 
julia> load(joinpath(poly_dir, "polydata"); serializer=Oscar.Serialization.MultiFileRefSerializer())
_a*y - z

With gzip compression:

julia> Qx, x = QQ[:x];

julia> F, a = number_field(x^2 + 2);

julia> R, (y, z) = F[:y, :z];

julia> p = a * y - z;

julia> poly_dir = mkdir("poly_dir");

julia> save(joinpath(poly_dir, "polydata"), p; serializer=Oscar.Serialization.MultiFileRefSerializer(), compression=:gzip)

julia> readdir(poly_dir)
4-element Vector{String}:
 "polydata.mrdi.gz"
 "polydata_52e4c5e2-ff94-4b1c-9832-a61d8a54331a.mrdi.gz"
 "polydata_c8be128f-a28c-4d08-a67f-5e1c2ad9409d.mrdi.gz"
 "polydata_f2bd8b4b-e6a7-4961-943b-1d68306889a2.mrdi.gz"

LPSerializer

Specialized for LinearProgram{QQFieldElem}. Writes the LP data to an external .lp file (the standard LP file format) and stores only a reference to it in the .mrdi file. The basepath argument is used as the filename prefix for the .lp file.

julia> P = cube(3);

julia> LP = linear_program(P, [3, -2, 4]; k=2, convention=:min);

julia> lp_dir = mkdir("lp_dir");

julia> serializer = Oscar.Serialization.LPSerializer(joinpath(lp_dir, "lp"));

julia> save(joinpath(lp_dir, "lp.mrdi"), LP; serializer=serializer);

julia> readdir(lp_dir)
2-element Vector{String}:
 "lp.mrdi"
 "lp_10612199771096508645.lp"

Objects that can be serialized

In this section we will list a selection of objects that may be (de-)serialized.

Many low level objects may be stored and these in turn allow serializing higher level objects. Such low level objects are various types of matrices, vectors and sets.

Combinatorics

Graph
SimplicialComplex

Commutative Algebra

Ideal
PolyRing
PolyRingElem
MPolyRing
MPolyRingElem

Groups

FPGroup
FinGenAbGroup
PcGroup
PermGroup
SubFPGroup
SubPcGroup

Polyhedral Geometry

Cone
LinearProgram
PolyhedralFan
PolyhedralComplex
Polyhedron
SubdivisionOfPoints

Toric Geometry

NormalToricVariety
ToricDivisor

Tropical Geometry

TropicalCurve
TropicalHypersurface

Listing all serializable types of the current session

If you are curious about whether your type can already be serialized given your version of Oscar you can run the following command in your active session.

julia> Oscar.Serialization.reverse_type_mapDict{String, Union{Dict{String, Type}, Type}} with 193 entries:
  "AbstractLieAlgebraElem"   => AbstractLieAlgebraElem
  "SmallTreeModel"           => SmallTreeModel
  "Vector"                   => Vector
  "MPolyPowersOfElement"     => MPolyPowersOfElement
  "FinGenAbGroupElem"        => FinGenAbGroupElem
  "SMatSpace"                => SMatSpace
  "RootSystem"               => RootSystem
  "Orderings.SymbOrdering"   => SymbOrdering
  "MPolyQuoRing"             => MPolyQuoRing
  "FinGenAbGroupHom"         => FinGenAbGroupHom
  "DualRootSpaceElem"        => DualRootSpaceElem
  "FiniteField"              => Dict{String, Type}("FqField"=>FqField, "fpField…
  "LieAlgebraModuleElem"     => LieAlgebraModuleElem
  "MPolyQuoLocalizedRingHom" => MPolyQuoLocalizedRingHom
  "Hecke.QuadSpace"          => QuadSpace
  "WeylGroupElem"            => WeylGroupElem
  "AcbFieldElem"             => AcbFieldElem
  "DirectSumLieAlgebra"      => DirectSumLieAlgebra
  "LieAlgebraModule"         => LieAlgebraModule
  ⋮                          => ⋮