API¶
User API¶
DataSetReader class¶
-
class DataSetReader¶
General purpose reader for data described by an Fides data model.
fides::io::DataSetReader
reads data described by an Fides data model and creates VTK-m datasets. See the Fides schema definition for the supported data model.DataSetReader
also supports reading meta-data.Public Types
-
enum DataModelInput¶
Input types when setting up the DataSetReader
Values:
-
enumerator JSONFile¶
Brief input is path to a JSON file with the data model
-
enumerator JSONString¶
Brief input is JSON containing the data model stored in a string
-
enumerator BPFile¶
Brief input is a BP file that contains attributes that provide details for the predefined data model to be generated by Fides
-
enumerator JSONFile¶
Public Functions
-
DataSetReader(const std::string &dataModel, DataModelInput inputType = DataModelInput::JSONFile, const Params ¶ms = Params(), bool createSharedPoints = false)¶
Constructor to set up the Fides reader
See also
- Parameters
dataModel – the value should be 1) a path to a JSON file describing the data model to be used by the reader, 2) a string containing valid JSON, 3) a path to a BP file containing attributes that Fides can use to generate a data model, or 4) a path to a BP file that contains a fides/schema attribute that contains the full JSON for the data model.
inputType – specifies what is stored in the dataModel arg. Optional
params – a map of ADIOS engine parameters to be used for each data source. Optional
-
void SetDataSourceParameters(const std::string &source, const DataSourceParams ¶ms)¶
Sets the parameters for a given data source. Currently, only the inline engine requires this to be called, which must happen before attempting to read data.
- Parameters
source – name of the
DataSource
, which should match a data_sources name given in the data model JSON.params – a map of parameters and their values
-
void SetDataSourceIO(const std::string &source, void *io)¶
Set the IO for a given
source
. This call should only be used when using the inline engine and must be called before attempting to read data or metadata.- Parameters
source – name of the
DataSource
, which should match a data_sources name given in the data model JSON.io – pointer to the ADIOS IO object
-
void SetDataSourceIO(const std::string &source, const std::string &io)¶
Set the IO for a given
source
. This call should only be used when using the inline engine and must be called before attempting to read data or metadata.- Parameters
source – name of the
DataSource
, which should match a data_sources name given in the data model JSON.io – the address to an ADIOS IO object, stored in a string
-
fides::metadata::MetaData ReadMetaData(const std::unordered_map<std::string, std::string> &paths, const std::string &groupName = "")¶
Read and return meta-data. This includes information such as the number of blocks, available fields etc.
- Parameters
paths – a map that provides the paths (filenames usually) corresponding to each data source.
groupName – looks for metadata on variables inside the given group.
-
vtkm::cont::PartitionedDataSet ReadDataSet(const std::unordered_map<std::string, std::string> &paths, const fides::metadata::MetaData &selections)¶
Read and return heavy-data.
- Parameters
paths – a map that provides the paths (filenames usually) corresponding to each data source.
selections – provides support for reading a subset of the data by providing choices for things such as time and blocks.
-
StepStatus PrepareNextStep(const std::unordered_map<std::string, std::string> &paths)¶
When reading in streaming mode, this method has to be called before reading any meta-data or heavy data. It will also move the reader to the next step. Fides will loop on a data source while ADIOS reports that it is NotReady, but the user should also check the return which could return fides::StepStatus::OK or fides::StepStatus::EndOfStream. If EndOfStream, all steps have been read.
- Parameters
paths – a map that provides the paths (filenames usually) corresponding to each data source.
- FIDES_DEPRECATED_SUPPRESS_BEGIN vtkm::cont::PartitionedDataSet ReadStep (const std::unordered_map< std::string, std::string > &paths, const fides::metadata::MetaData &selections)
Same as
ReadDataSet
except that it works in streaming mode and needs to be preceeded by PrepareStep.- Parameters
paths – a map that provides the paths (filenames usually) corresponding to each data source.
selections – provides support for reading a subset of the data by providing choices for things such as time and blocks.
- FIDES_DEPRECATED_SUPPRESS_END FIDES_DEPRECATED_SUPPRESS_BEGIN std::shared_ptr< fides::datamodel::FieldDataManager > GetFieldData ()
Get a pointer to the field data manager
See also
FieldDataManager, FieldData
- FIDES_DEPRECATED_SUPPRESS_END std::vector< std::string > GetDataSourceNames ()
Get std::vector of DataSource names.
-
std::set<std::string> GetGroupNames(const std::unordered_map<std::string, std::string> &paths)¶
Get all available group names.
Public Static Functions
-
static bool CheckForDataModelAttribute(const std::string &filename, const std::string &attrName = "Fides_Data_Model")¶
Checks a bp file for an attribute containing information that Fides can use to generate the data model. Static so that it doesn’t require setting up the DataSetReader first. Useful for applications like ParaView, where it wants to check if it can use Fides to read a file without needing to configure Fides first.
- Parameters
filename – Name of file to check
attrName – Name of attribute to look for
-
enum DataModelInput¶
Keys and MetaData¶
-
KeyType fides::keys::NUMBER_OF_BLOCKS()¶
Key used for storing number of blocks meta-data. Uses fides::metadata::Size
-
KeyType fides::keys::NUMBER_OF_STEPS()¶
Key used for storing number of steps meta-data. Uses fides::metadata::Size
-
KeyType fides::keys::BLOCK_SELECTION()¶
Key used for selecting a set of blocks. Uses fides::metadata::Vector<size_t>
-
KeyType fides::keys::FIELDS()¶
Key used for available array meta-data and array selection. Uses fides::metadata::Vector<fides::metadata::FieldInformation>
-
KeyType fides::keys::STEP_SELECTION()¶
Key used for selecting time step. Uses fides::metadata::Index
-
KeyType fides::keys::PLANE_SELECTION()¶
Key used for selecting planes for XGC data. Should only be used internally. Uses fides::metadata::Set
-
struct Size : public fides::metadata::MetaDataItem¶
Meta-data item to store size of things such as number of blocks.
Public Functions
-
inline Size(size_t nItems)¶
constructor
Public Members
-
size_t NumberOfItems¶
Number of items (e.g., blocks)
-
inline Size(size_t nItems)¶
-
struct Index : public fides::metadata::MetaDataItem¶
Meta-data item to store an index to a container.
Public Functions
-
inline Index(size_t idx)¶
Public Members
-
size_t Data¶
-
inline Index(size_t idx)¶
-
struct FieldInformation¶
Simple struct representing field information.
Public Functions
-
inline FieldInformation(std::string name, vtkm::cont::Field::Association assoc)¶
- inline FIDES_DEPRECATED_SUPPRESS_BEGIN FIDES_DEPRECATED (1.1, "fides::Association is no longer used. Use vtkm::cont::Field::Association directly.") FieldInformation(std
Public Members
- FIDES_DEPRECATED_SUPPRESS_END std::string Name
Name of the field.
-
vtkm::cont::Field::Association Association¶
Association of the field. See VTK-m field association for details.
-
inline FieldInformation(std::string name, vtkm::cont::Field::Association assoc)¶
-
template<typename T>
struct Vector : public fides::metadata::MetaDataItem¶ Meta-data item to store a vector.
-
template<typename T>
struct Set : public fides::metadata::MetaDataItem¶ Meta-data item to store a set.
-
class MetaData¶
Container of meta-data items. This class is a simple wrapper around an std::map that makes setting/getting a bit easier. Internally, it stores objects using unique_ptrs but the interface uses stack objects.
Public Functions
-
template<typename T>
inline void Set(fides::keys::KeyType key, const T &item)¶ Add a meta-data item to the map. Supports subclasses of
MetaDataItem
only.
-
template<typename T>
inline const T &Get(fides::keys::KeyType key) const¶ Given a type, returns an object if it exists. Raises an exception if the item does not exist or if the provided template argument is incorrect.
-
inline void Remove(fides::keys::KeyType key)¶
Given a key, removes the item from the map.
-
inline bool Has(fides::keys::KeyType key) const¶
Given a key, checks whether an item exists.
-
template<typename T>
Useful enums and typedefs¶
-
enum fides::StepStatus¶
Possible return values when using Fides in a streaming mode.
Values:
-
enumerator OK¶
-
enumerator NotReady¶
-
enumerator EndOfStream¶
-
enumerator OK¶
-
using fides::DataSourceParams = std::unordered_map<std::string, std::string>¶
Parameters for an individual data source, e.g., Parameters needed by ADIOS for configuring an Engine.
-
using fides::Params = std::unordered_map<std::string, DataSourceParams>¶
Parameters for all data sources mapped to their source name. The key must match the name given for the data source in the JSON file.