API

User API

DataSetReader class

class fides::io::DataSetReader

General purpose reader for data described by an Fides data model.

fides::io::DataSetReader reads data described by an Fides data model and creates VTK-m datasets. See the Fides schema definition for the supported data model. DataSetReader also supports reading meta-data.

Public Types

enum DataModelInput

Input types when setting up the DataSetReader

Values:

enumerator JSONFile

Brief input is path to a JSON file with the data model

enumerator JSONString

Brief input is JSON containing the data model stored in a string

enumerator BPFile

Brief input is a BP file that contains attributes that provide details for the predefined data model to be generated by Fides

Public Functions

DataSetReader(const std::string dataModel, DataModelInput inputType = DataModelInput::JSONFile, const Params &params = Params())

Constructor to set up the Fides reader

See

DataModelInput

Parameters

• dataModel: the value should be 1) a path to a JSON file describing the data model to be used by the reader, 2) a string containing valid JSON, or 3) a path to a BP file containing attributes that Fides can use to generate a data model.

• inputType: specifies what is stored in the dataModel arg. Optional

• params: a map of ADIOS engine parameters to be used for each data source. Optional

void SetDataSourceParameters(const std::string source, const DataSourceParams &params)

Sets the parameters for a given data source. Currently, only the inline engine requires this to be called, which must happen before attempting to read data.

Parameters

• source: name of the DataSource, which should match a data_sources name given in the data model JSON.

• params: a map of parameters and their values

void SetDataSourceIO(const std::string source, void *io)

Set the IO for a given source. This call should only be used when using the inline engine and must be called before attempting to read data or metadata.

Parameters

• source: name of the DataSource, which should match a data_sources name given in the data model JSON.

• io: pointer to the ADIOS IO object

fides::metadata::MetaData ReadMetaData(const std::unordered_map<std::string, std::string> &paths)

Read and return meta-data. This includes information such as the number of blocks, available fields etc.

Parameters

• paths: a map that provides the paths (filenames usually) corresponding to each data source.

vtkm::cont::PartitionedDataSet ReadDataSet(const std::unordered_map<std::string, std::string> &paths, const fides::metadata::MetaData &selections)

Read and return heavy-data.

Parameters

• paths: a map that provides the paths (filenames usually) corresponding to each data source.

• selections: provides support for reading a subset of the data by providing choices for things such as time and blocks.

StepStatus PrepareNextStep(const std::unordered_map<std::string, std::string> &paths)

When reading in streaming mode, this method has to be called before reading any meta-data or heavy data. It will also move the reader to the next step. Fides will loop on a data source while ADIOS reports that it is NotReady, but the user should also check the return which could return fides::StepStatus::OK or fides::StepStatus::EndOfStream. If EndOfStream, all steps have been read.

Parameters

• paths: a map that provides the paths (filenames usually) corresponding to each data source.

vtkm::cont::PartitionedDataSet ReadStep(const std::unordered_map<std::string, std::string> &paths, const fides::metadata::MetaData &selections)

Same as ReadDataSet except that it works in streaming mode and needs to be preceeded by PrepareStep.

Parameters

• paths: a map that provides the paths (filenames usually) corresponding to each data source.

• selections: provides support for reading a subset of the data by providing choices for things such as time and blocks.

std::shared_ptr<fides::datamodel::FieldDataManager> GetFieldData()

Get a pointer to the field data manager

See

FieldDataManager, FieldData

std::vector<std::string> GetDataSourceNames()

Get std::vector of DataSource names.

Public Static Functions

bool CheckForDataModelAttribute(const std::string &filename, const std::string &attrName = "Fides_Data_Model")

Checks a bp file for an attribute containing information that Fides can use to generate the data model. Static so that it doesn’t require setting up the DataSetReader first. Useful for applications like ParaView, where it wants to check if it can use Fides to read a file without needing to configure Fides first.

Parameters

• filename: Name of file to check

• attrName: Name of attribute to look for

class DataSetReaderImpl

Keys and MetaData

KeyType fides::keys::NUMBER_OF_BLOCKS()

Key used for storing number of blocks meta-data. Uses fides::metadata::Size

KeyType fides::keys::NUMBER_OF_STEPS()

Key used for storing number of steps meta-data. Uses fides::metadata::Size

KeyType fides::keys::BLOCK_SELECTION()

Key used for selecting a set of blocks. Uses fides::metadata::Vector<size_t>

KeyType fides::keys::FIELDS()

Key used for available array meta-data and array selection. Uses fides::metadata::Vector<fides::metadata::FieldInformation>

KeyType fides::keys::STEP_SELECTION()

Key used for selecting time step. Uses fides::metadata::Index

KeyType fides::keys::PLANE_SELECTION()

Key used for selecting planes for XGC data. Should only be used internally. Uses fides::metadata::Set

struct fides::metadata::Size : public fides::metadata::MetaDataItem

Meta-data item to store size of things such as number of blocks.

Public Functions

Size(size_t nItems)

constructor

Public Members

size_t NumberOfItems

Number of items (e.g., blocks)

struct fides::metadata::Index : public fides::metadata::MetaDataItem

Meta-data item to store an index to a container.

Public Functions

Index(size_t idx)

Public Members

size_t Data

struct fides::metadata::FieldInformation

Simple struct representing field information.

Public Functions

FieldInformation(std::string name, vtkm::cont::Field::Association assoc)

FieldInformation(std::string name, fides::Association assoc)

Public Members

std::string Name

Name of the field.

fides::Association Association

Association of the field.

See

fides::Association

template<typename T>
struct fides::metadata::Vector : public fides::metadata::MetaDataItem

Meta-data item to store a vector.

Public Functions

Vector(std::vector<T> vec)

Vector()

Public Members

std::vector<T> Data

template<typename T>
struct fides::metadata::Set : public fides::metadata::MetaDataItem

Meta-data item to store a set.

Public Functions

Set(std::set<T> data)

Set()

Public Members

std::set<T> Data

class fides::metadata::MetaData

Container of meta-data items. This class is a simple wrapper around an std::map that makes setting/getting a bit easier. Internally, it stores objects using unique_ptrs but the interface uses stack objects.

Public Functions

template<typename T>
void Set(fides::keys::KeyType key, const T &item)

Add a meta-data item to the map. Supports subclasses of MetaDataItem only.

template<typename T>
const T &Get(fides::keys::KeyType key) const

Given a type, returns an object if it exists. Raises an exception if the item does not exist or if the provided template argument is incorrect.

void Remove(fides::keys::KeyType key)

Given a key, removes the item from the map.

bool Has(fides::keys::KeyType key) const

Given a key, checks whether an item exists.

FieldData related classes

class fides::datamodel::FieldData

Class to store data that does not have an Association of points or cells.

Data is stored in VariantArrayHandles, with one VariantArrayHandle per data block.

Public Functions

std::string GetName() const

Returns the name of this field.

const std::vector<vtkm::cont::VariantArrayHandle> &GetData() const

Get a reference to the data. Each element of the vector stores one block.

std::vector<vtkm::cont::VariantArrayHandle> &GetData()

Get a reference to the data. Each element of the vector stores one block.

class fides::datamodel::FieldDataManager

Stores all FieldData.

Use this to access fields that are marked as fides::Association::FIELD_DATA.

Public Functions

void AddField(const std::string &name, FieldData array)

Add the given FieldData to the Manager. Throws error if field already exists.

Parameters

• name: name of field to add

• array: FieldData object to add

bool HasField(const std::string &name)

Check to see if the Manager already contains FieldData with the given name.

Parameters

• name: name of field

FieldData &GetField(const std::string &name)

Returns the FieldData stored with the given name. Throws an error if the field isn’t found.

Parameters

• name: name of field

const std::unordered_map<std::string, FieldData> &GetAllFields()

Returns a ref to the unordered_map containing all fields.

Useful enums and typedefs

enum fides::StepStatus

Possible return values when using Fides in a streaming mode.

Values:

enumerator OK

enumerator NotReady

enumerator EndOfStream

enum fides::Association

Association for fields, based on VTK-m’s association enum, but also includes a value for representing field data.

Values:

enumerator POINTS

enumerator CELL_SET

enumerator FIELD_DATA

using fides::DataSourceParams = std::unordered_map<std::string, std::string>

Parameters for an individual data source, e.g., Parameters needed by ADIOS for configuring an Engine.

using fides::Params = std::unordered_map<std::string, DataSourceParams>

Parameters for all data sources mapped to their source name. The key must match the name given for the data source in the JSON file.