Implementation of an importer

This topic introduces:

Basic concepts about the implementation and usage of custom importers used in conjunction with the Iray API:
- Implementing an importer
- Registering an importer
An example importer program consisting of vanilla_importer.h and example_importer.cpp. The importer is called Vanilla. For simplicity, it is defined in and used by the main application. The importer is intended as an illustrative skeleton: It implements all interfaces but does not parse the file content in a meaningful way.

Implementing an importer

The implementation of the Vanilla importer is structured in three parts:

Implementing the mi::neuraylib::IImpexp_state interface
Implementing the mi::neuraylib::IImporter interface
Implementing the mi::neuraylib::IImporter::import_elements() method

Instances of mi::neuraylib::IImpexp_state are used to pass information about the current importer state, for example to recursive calls of importers. The Vanilla importer does not need to carry around any additional information besides what is required by the interface, therefore this simple implementation is fine. Note that the simple derivation from mi::base::Interface_implement suffices here (in contrast to example_plugins.cpp).

The Vanilla importer is given in the implementation of the mi::neuraylib::IImporter interface. Later, an instance of this class is registered as an importer with the Iray API. Most of the methods implemented here are defined in the base interface mi::neuraylib::IImpexp_state, as they are common for importers and exporters. The Vanilla importer claims to handle files with the extension .vnl and .van. It does not require specific capabilities of the reader to handle these formats. However, if the reader supports the lookahead capability, it will use a magic header check instead of relying on file name extensions.

The actual work of the Vanilla importer occurs in the import_elements() method. It is split into three parts:

Creating an import result object
Creating a group object which acts as the root group
Reading the file line by line

While performing these tasks, the example program demonstrates what type of errors to detect, how errors can be reported, and how to implement an include file mechanism, and similar things, with a recursive call to the mi::neuraylib::IImport_api::import_elements() method.

Registering an importer

Registering importers is similar to registering user-defined classes. However, since importers are different from regular classes (for example, you cannot create instances of them using mi::neuraylib::ITransaction::create()); you need to use a registration method specific to importers. This registration method expects a pointer to an instance of the custom importer.

To run the example, you need to create two files called test1.vnl and test2.van, each five lines long and starting with a line saying VANILLA. For demonstration purposes, the example will print the generated error messages and the names of the imported elements.

vanilla_importer.h

Plain text

001 /******************************************************************************
002  * © 1986, 2016 NVIDIA Corporation. All rights reserved.
003  *****************************************************************************/
004 
005 #include <mi/neuraylib.h>
006 
007 #include <string>
008 #include <sstream>
009 
010 // Support function to handle mi::base::Message_severity to std::string conversion
011 static std::string enum_to_str( mi::base::Message_severity severity)
012 {
013     switch( severity) {
014         case mi::base::MESSAGE_SEVERITY_FATAL:   return "fatal";
015         case mi::base::MESSAGE_SEVERITY_ERROR:   return "error";
016         case mi::base::MESSAGE_SEVERITY_WARNING: return "warning";
017         case mi::base::MESSAGE_SEVERITY_INFO:    return "information";
018         case mi::base::MESSAGE_SEVERITY_VERBOSE: return "verbose";
019         case mi::base::MESSAGE_SEVERITY_DEBUG:   return "debug";
020         default:                                 return "unknown" ;
021     }
022 }
023 
024 // Define importer state, which is used in the importer function to carry additional data around for
025 // possible recursive invocations of importers. It contains a URI for the imported resource, a line
026 // number, and a pointer to a parent state supporting recursive imports.
027 class Vanilla_import_state
028   : public mi::base::Interface_implement< mi::neuraylib::IImpexp_state>
029 {
030 public:
031     // Constructor.
032     Vanilla_import_state( const char* uri, const mi::neuraylib::IImpexp_state* parent_state)
033       : m_line_number( 1),
034         m_uri( uri ? uri : ""),
035         m_parent_state( parent_state, mi::base::DUP_INTERFACE) { }
036 
037     // Definition of all interface functions.
038 
039     const char* get_uri() const { return m_uri.empty() ? 0 : m_uri.c_str(); }
040 
041     mi::Uint32 get_line_number() const { return m_line_number; }
042 
043     void set_line_number( mi::Uint32 number) { m_line_number = number; }
044 
045     void incr_line_number() { ++m_line_number; }
046 
047     const mi::neuraylib::IImpexp_state* get_parent_state() const
048     {
049         if( m_parent_state)
050             m_parent_state->retain();
051         return m_parent_state.get();
052     }
053 
054 private:
055     mi::Uint32 m_line_number;
056     std::string m_uri;
057     mi::base::Handle<const mi::neuraylib::IImpexp_state> m_parent_state;
058 };
059 
060 // Define importer. It defines all meta information, for example, author, version numbers, which
061 // formats it supports etc. The longer format detection functions and the import_elements() function
062 // are implemented outside of the class body.
063 class Vanilla_importer
064   : public mi::base::Interface_implement< mi::neuraylib::IImporter>
065 {
066 public:
067     // Constructor.
068     Vanilla_importer( mi::neuraylib::IPlugin_api* plugin_api)
069         : m_plugin_api( plugin_api, mi::base::DUP_INTERFACE) { }
070 
071     // Definition of all interface functions.
072 
073     mi::neuraylib::IImpexp_state* create_impexp_state(
074         const char* uri, const mi::neuraylib::IImpexp_state* parent_state) const
075     {
076         return new Vanilla_import_state( uri, parent_state);
077     }
078 
079     // This importer supports the file name extensions ".vnl" and ".van".
080     const char* get_supported_extensions( mi::Uint32 i) const
081     {
082         switch( i) {
083             case 0:  return ".vnl";
084             case 1:  return ".van";
085             default: return 0;
086         }
087     }
088 
089     mi::neuraylib::Impexp_priority get_priority() const
090     {
091         return mi::neuraylib::IMPEXP_PRIORITY_WELL_DEFINED;
092     }
093 
094     const char* get_name() const { return "NVIDIA example vanilla (v1) importer"; }
095 
096     const char* get_author() const { return "NVIDIA ARC GmbH, Berlin, Germany"; }
097 
098     mi::base::Uuid get_uuid() const
099     {
100         mi::base::Uuid uuid;
101         uuid.m_id1 = 0x338eca60;
102         uuid.m_id2 = 0x31004802;
103         uuid.m_id3 = 0xaab9046b;
104         uuid.m_id4 = 0x9e0b1d9b;
105         return uuid;
106     }
107 
108     mi::Uint32 get_major_version() const { return 1; }
109 
110     mi::Uint32 get_minor_version() const { return 0; }
111 
112     bool test_file_type( const char* extension) const;
113 
114     bool test_file_type( const char* extension, const mi::neuraylib::IReader* reader) const;
115 
116     mi::neuraylib::IImport_result* import_elements(
117         mi::neuraylib::ITransaction* transaction,
118         const char* extension,
119         mi::neuraylib::IReader* reader,
120         const mi::IMap* options,
121         mi::neuraylib::IImpexp_state* import_state) const;
122 
123 private:
124     // Definition of support functions. They are not part of the interface.
125 
126     // Format message with context and append it to the messages in the result.
127     static mi::neuraylib::IImport_result_ext* report_message(
128         mi::neuraylib::IImport_result_ext* result,
129         mi::Uint32 message_number,
130         mi::base::Message_severity message_severity,
131         std::string message,
132         const mi::neuraylib::IImpexp_state* import_state)
133     {
134         std::ostringstream s;
135         s << import_state->get_uri()
136           << ":" << import_state->get_line_number() << ": "
137           << "Vanilla importer message " << message_number << ", "
138           << "severity " << enum_to_str( message_severity) << ": "
139           << message;
140         // Report context of all parent import states from recursive invocations of
141         // import_elements() in their own lines with indentation.
142         mi::base::Handle<const mi::neuraylib::IImpexp_state> parent_state(
143             import_state->get_parent_state());
144         while( parent_state.is_valid_interface()) {
145             s << "\n    included from: " << parent_state->get_uri()
146               << ":" << parent_state->get_line_number();
147             parent_state = parent_state->get_parent_state();
148         }
149         result->message_push_back( message_number, message_severity, s.str().c_str());
150         return result;
151     }
152 
153     mi::base::Handle<mi::neuraylib::IPlugin_api> m_plugin_api;
154 };
155 
156 bool Vanilla_importer::test_file_type( const char* extension) const
157 {
158     // This importer supports the file name extensions ".vnl" and ".van".
159     mi::Size len = std::strlen( extension);
160     return (len > 3)
161         &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
162           || ( 0 == strcmp( extension + len - 4, ".van")));
163 }
164 
165 bool Vanilla_importer::test_file_type(
166     const char* extension, const mi::neuraylib::IReader* reader) const
167 {
168     // Use magic header check if lookahead is available
169     if( reader->supports_lookahead()) {
170         // File has to start with "VANILLA" and linebreak, which can be \n or \r depending on the
171         // line ending convention in the file.
172         const char** buffer = 0;
173         mi::Sint64 n = reader->lookahead( 8, buffer);
174         return ( n >= 8) && (0 == std::strncmp( *buffer, "VANILLA", 7))
175                          && ((*buffer[7] == '\n') || (*buffer[7] == '\r'));
176     }
177     // This importer supports the file name extensions ".vnl" and ".van".
178     mi::Size len = std::strlen( extension);
179     return (len > 3)
180         &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
181           || ( 0 == strcmp( extension + len - 4, ".van")));
182 }
183 
184 mi::neuraylib::IImport_result* Vanilla_importer::import_elements(
185     mi::neuraylib::ITransaction* transaction,
186     const char* extension,
187     mi::neuraylib::IReader* reader,
188     const mi::IMap* importer_options,
189     mi::neuraylib::IImpexp_state* import_state) const
190 {
191     // Create the importer result instance for the return value. If that fails something is really
192     // wrong and we return 0.
193     mi::neuraylib::IImport_result_ext* result
194         = transaction->create<mi::neuraylib::IImport_result_ext>( "Import_result_ext");
195     if( !result)
196         return 0;
197 
198     // Get the 'prefix' option.
199     MISTD::string prefix;
200     if( importer_options && importer_options->has_key( "prefix")) {
201         mi::base::Handle<const mi::IString> option(
202             importer_options->get_value<mi::IString>( "prefix"));
203         prefix = option->get_c_str();
204     }
205 
206     // Get the 'list_elements' option.
207     bool list_elements = false;
208     if( importer_options && importer_options->has_key( "list_elements")) {
209         mi::base::Handle<const mi::IBoolean> option(
210             importer_options->get_value<mi::IBoolean>( "list_elements"));
211         list_elements = option->get_value<bool>();
212     }
213 
214     // Before we start parsing the file, we create a group that collects all top-level elements.
215     // This will be our rootgroup.
216     std::string root_group_name = prefix + "Vanilla::root_group";
217     mi::base::Handle<mi::neuraylib::IGroup> rootgroup(
218         transaction->create<mi::neuraylib::IGroup>( "Group"));
219     mi::Sint32 error_code = transaction->store( rootgroup.get(), root_group_name.c_str());
220     if( error_code != 0)
221         return report_message( result, 4010, mi::base::MESSAGE_SEVERITY_ERROR,
222             "failed to create the root group", import_state);
223     rootgroup = transaction->edit<mi::neuraylib::IGroup>( root_group_name.c_str());
224 
225     // Register the rootgroup with the importer result.
226     result->set_rootgroup( root_group_name.c_str());
227 
228     // If the element list flag is set, record the rootgroup element also in the elements array of
229     // the result.
230     if( list_elements)
231         result->element_push_back( root_group_name.c_str());
232 
233     // Assume it is a line based text format and read it line by line. Assume lines are no longer
234     // than 256 chars, otherwise read it in pieces
235     char buffer[257];
236     while( reader->readline( buffer, 257) && buffer[0] != '\0') {
237 
238         // Periodically check whether the transaction is still open. If not, stop importing.
239         if( !transaction->is_open())
240             break;
241 
242         // Here you can process the buffer content of size len. It corresponds to the line
243         // import_state->get_line_number() of the input file.
244         mi::Size len = std::strlen( buffer);
245 
246         // We illustrate some actions triggered by fixed line numbers: Line 3 of a ".vnl" file
247         // triggers the recursive inclusion of the file "test2.van" file with a prefix.
248         mi::Size ext_len = std::strlen( extension);
249         if( 3 == import_state->get_line_number()
250              && (ext_len > 3)
251              && 0 == strcmp( extension + ext_len - 4, ".vnl")) {
252             // Get a IImport_api handle to call its import_elements() function
253             mi::base::Handle<mi::neuraylib::IImport_api> import_api(
254                 m_plugin_api->get_api_component<mi::neuraylib::IImport_api>());
255             if( !import_api.is_valid_interface())
256                 // Error numbers from 4000 to 5999 are reserved for custom importer messages like
257                 // this one
258                 return report_message( result, 4001, mi::base::MESSAGE_SEVERITY_ERROR,
259                    "did not get a valid IImport_api object, import failed", import_state);
260             // Call the importer recursively to illustrate the handling of include files and similar
261             // things. We trigger this import only on a ".vnl" file and include the fixed file
262             // "test2.van". We give the included file an extra prefix "Prefix_".
263             mi::base::Handle<mi::neuraylib::IFactory> factory(
264                 m_plugin_api->get_api_component<mi::neuraylib::IFactory>());
265             mi::base::Handle<mi::IString> child_prefix( factory->create<mi::IString>( "String"));
266             child_prefix->set_c_str( "Prefix_");
267             mi::base::Handle<mi::IMap> child_importer_options( factory->clone<mi::IMap>(
268                 importer_options));
269             child_importer_options->erase( "prefix");
270             child_importer_options->insert( "prefix", child_prefix.get());
271             mi::base::Handle<const mi::neuraylib::IImport_result> include_result(
272                 import_api->import_elements( transaction, "file:test2.van",
273                     child_importer_options.get(), import_state));
274             // Safety check, if this fails, the import is not continued.
275             if( !include_result.is_valid_interface())
276                 return report_message( result, 4002, mi::base::MESSAGE_SEVERITY_ERROR,
277                    "import was not able to create result object, import failed", import_state);
278             // Process the result. Even in the case of an error, we need to process the elements
279             // array.
280             if( list_elements)
281                 result->append_elements( include_result.get());
282             // Append messages of include to this result
283             result->append_messages( include_result.get());
284             // React on errors during processing of the include
285             if( include_result->get_error_number() > 0) {
286                 // Report the failure of the include as a separate message too
287                 report_message( result, 4003, mi::base::MESSAGE_SEVERITY_ERROR,
288                     "including file 'test2.van' failed.", import_state);
289             } else {
290                 // Recursive import was successful. The rootgroup of the import is now appended to
291                 // this rootgroup
292                 if( 0 == include_result->get_rootgroup())
293                     report_message( result, 4004, mi::base::MESSAGE_SEVERITY_ERROR,
294                         "include file 'test2.van' did not contain a rootgroup", import_state);
295                 else
296                     rootgroup->attach( include_result->get_rootgroup());
297             }
298         }
299 
300         // Line 4 triggers several messages and adds an empty group to the rootgroup.
301         if( 4 == import_state->get_line_number()) {
302 
303             // Several messages, file parsing continues
304             report_message( result, 4005, mi::base::MESSAGE_SEVERITY_FATAL,
305                 "test message in line 4", import_state);
306             report_message( result, 4006, mi::base::MESSAGE_SEVERITY_ERROR,
307                 "test message in line 4", import_state);
308             report_message( result, 4007, mi::base::MESSAGE_SEVERITY_WARNING,
309                 "test message in line 4", import_state);
310             report_message( result, 4008, mi::base::MESSAGE_SEVERITY_INFO,
311                 "test message in line 4", import_state);
312             report_message( result, 4009, mi::base::MESSAGE_SEVERITY_VERBOSE,
313                  "test message in line 4", import_state);
314 
315             // Create a group "Vanilla::Group1"
316             std::string group_name = prefix + "Vanilla::Group1";
317             mi::base::Handle<mi::neuraylib::IGroup> group(
318                 transaction->create<mi::neuraylib::IGroup>( "Group"));
319             mi::Sint32 error_code = transaction->store( group.get(), group_name.c_str());
320             if( error_code != 0)
321                 report_message( result, 4011, mi::base::MESSAGE_SEVERITY_ERROR,
322                     "unexpected error in line 4", import_state);
323             else {
324                 // Add this group to the rootgroup
325                 rootgroup->attach( group_name.c_str());
326                 // If get_list_elements_flag is set, record the new element in the elements array of
327                 // the result.
328                 if( list_elements)
329                     result->element_push_back( group_name.c_str());
330             }
331         }
332 
333         // Handle line numbers, buffer might end in '\n' or not if line was too long.
334         if( (len > 0) && ('\n' == buffer[len-1]))
335             import_state->incr_line_number();
336     }
337 
338     if( reader->eof()) {
339         // Normal end
340         return result;
341     }
342 
343     // Report error condition for a failed reader call
344     return report_message(
345         result,
346         static_cast<mi::Uint32>( reader->get_error_number()),
347         mi::base::MESSAGE_SEVERITY_ERROR,
348         reader->get_error_message() ? reader->get_error_message() : "",
349         import_state);
350 }

example_importer.cpp