Example for Importers

• Implementation of an importer

• Registration of an importer

Implementation of an importer

The implementation of the Vanilla importer in the example source in structured in three parts:

• Implementation of the INVALID_DOXYREFmi::IImpexp_state interface

• Implementation of the mi::neuraylib::IImporter interface

• Implementation of the mi::neuraylib::IImporter::import_elements() method

Instances of INVALID_DOXYREFmi::IImpexp_state are used to pass information about the current importer state, for example to recursive calls of importers. The Vanilla importer does not need to carry around any additional information besides what is required by the interface, therefore this simple implementation is fine. Note that the simple derivation from mi::base::Interface_implement suffices here (in contrast to user-defined classes).

The Vanilla importer is given in the implementation of the mi::neuraylib::IImporter interface. Later, an instance of this class will be registered as importer with the neuray API. Most of the methods implemented here are actually defined in the base interface INVALID_DOXYREFmi::IImpexp_state, as they are common for importers and exporters. The Vanilla importer claims to handle files with extension ".vnl" and ".van". It does not require specific capabilities of the reader to handle these formats. However, if the readers supports the lookahead capability, it will use a magic header check instead of relying on file name extensions.

The actual work of the Vanilla importer happens in the import_elements() method. It is split into three parts:

• creation of the import result object

• creation of a group object which acts as rootgroup

• reading the file line by line

While performing these tasks the example demonstrates what type of errors to detect, a way to report the errors, and how to implement an include file mechanism and similar things with a recursive call to the mi::neuraylib::IImport_api::import_elements() method.

Registration of an importer

The registration of an importer is similar to the registration of user-defined classes. However, since importers are different from regular classes (e.g., you cannot create instances of them using mi::neuraylib::ITransaction::create()) you need to use a registration method specific to importers. This registration method expects a pointer to an instance of the custom importer.

To run the example, you need to create two files called test1.vnl and test2.van, each 5 lines long and starting with a line saying VANILLA. The example will print the (for demonstration purposes generated) errors messages and the names of the imported elements.

‎/******************************************************************************
 * Copyright 1986, 2011 NVIDIA Corporation. All rights reserved.
 *****************************************************************************/

#include <mi/neuraylib.h>

#include <string>
#include <sstream>

// Support function to handle mi::base::Message_severity to std::string conversion
static std::string enum_to_str( mi::base::Message_severity severity)
{
    switch( severity) {
        case mi::base::MESSAGE_SEVERITY_FATAL:
            return "fatal";
        case mi::base::MESSAGE_SEVERITY_ERROR:
            return "error";
        case mi::base::MESSAGE_SEVERITY_WARNING:
            return "warning";
        case mi::base::MESSAGE_SEVERITY_INFO:
            return "information";
        case mi::base::MESSAGE_SEVERITY_VERBOSE:
            return "verbose";
        default:
            return "unknown" ;
    }
}

// Define importer state, which is used in the importer function to carry additional data around
// for possible recursive invocations of importers. It contains a URI for the imported resource,
// a line number, and a pointer to a parent state supporting recursive imports.
class Vanilla_import_state
    : public mi::base::Interface_implement< mi::IImpexp_state>
{
    // Member variables to keep all necessary data
    std::string               m_uri;
    mi::Uint32                m_line_number;
    const INVALID_DOXYREFmi::IImpexp_state*  m_parent_state;

public:
    // Definition of all interface functions.

    const char* get_uri() const { return m_uri.empty() ? 0 : m_uri.c_str(); }

    mi::Uint32 get_line_number() const { return m_line_number; }

    void set_line_number( mi::Uint32 number) { m_line_number = number; }

    void incr_line_number() { ++m_line_number; }

    const INVALID_DOXYREFmi::IImpexp_state* get_parent_state() const { return m_parent_state; }

    // Definition of corresponding setters / constructors. They are not part of the interface.

    // Default constructor, initializes line number to 1.
    Vanilla_import_state()
        : m_line_number( 1),
          m_parent_state( 0)
    {}

    void set_uri( const char* uri) { m_uri = uri ? uri : ""; }

    void set_parent_state( const INVALID_DOXYREFmi::IImpexp_state* parent_state)
    {
        m_parent_state = parent_state;
    }
};

// Define importer. It defines all meta information, for example, author, version numbers,
// which formats it supports etc. The longer format detection functions and import_elements
// function are implemented outside of the class body.
class Vanilla_importer
    : public mi::base::Interface_implement< mi::neuraylib::IImporter>
{
    mi::neuraylib::IPlugin_api* m_iplugin_api;

public:
    // Returns a state suitable for passing it to an import call.
    // The parameters are used to initialize the corresponding properties of the state.
    // The line number is set to 1.
    INVALID_DOXYREFmi::IImpexp_state* create_impexp_state (
        const char* uri,
        const INVALID_DOXYREFmi::IImpexp_state* parent_state) const
    {
        Vanilla_import_state* import_state = new Vanilla_import_state();
        import_state->set_uri( uri);
        import_state->set_parent_state( parent_state);
        return import_state;
    }

    // This importer supports the file name extensions ".vnl" and ".van".
    const char* get_supported_extensions( mi::Uint32 i) const
    {
        switch( i) {
            case 0:  return ".vnl";
            case 1:  return ".van";
            default: return 0;
        }
    }

    // Returns the confidence of the importer that its test_file_type() can identify the file and
    // that the file format is fully supported.
    INVALID_DOXYREFmi::Impexp_priority get_priority () const
    {
        return INVALID_DOXYREFmi::IMPEXP_PRIORITY_WELL_DEFINED;
    }

    // Returns a concise single-line clear text description of the importer.
    const char* get_name () const
    {
        return "mental images example vanilla (v1) importer";
    }

    // Returns a concise single-line clear text description of the author of
    // this importer.
    const char* get_author () const
    {
        return "mental images GmbH, Berlin, Germany";
    }

    // Returns the unique identifier for the importer.
    mi::base::Uuid get_uuid() const
    {
        mi::base::Uuid uuid;
        uuid.m_id1 = 0x338eca60;
        uuid.m_id2 = 0x31004802;
        uuid.m_id3 = 0xaab9046b;
        uuid.m_id4 = 0x9e0b1d9b;
        return uuid;
    }

    // Returns the major version number of the importer.
    mi::Uint32 get_major_version() const { return 1; }

    // Returns the minor version number of the importer.
    mi::Uint32 get_minor_version() const { return 0; }

    // Returns true if the importer can handle the file type determined by the file name extension.
    bool test_file_type ( const char* extension) const;

    // Returns true if the importer can handle the file type determined by the file name extension
    // and if the reader has sufficient capabilities for import.
    bool test_file_type( const char* extension,
                                 const INVALID_DOXYREFmi::IReader* reader) const;

    // Imports all elements from the reader in a format determined by the file extension and
    // (optionally) the lookahead of the reader.
    INVALID_DOXYREFmi::IImport_result* import_elements (
        mi::neuraylib::ITransaction* transaction,
        const char*                  extension,
        INVALID_DOXYREFmi::IReader*                 reader,
        const mi::IMap*              options,
        INVALID_DOXYREFmi::IImpexp_state*           import_state) const;

    // Definition of constructors and support functions. They are not part of the interface.

    // Constructor.
    Vanilla_importer( mi::neuraylib::IPlugin_api* iplugin_api)
        : m_iplugin_api( iplugin_api)
    {
        m_iplugin_api->retain();
    }

    // Destructor.
    ~Vanilla_importer()
    {
        m_iplugin_api->release();
    }

    // Format error message with context and append it to the error messages in the result.
    static INVALID_DOXYREFmi::IImport_result_ext* report_error(
        mi::neuraylib::ITransaction* transaction,
        INVALID_DOXYREFmi::IImport_result_ext*      result,
        mi::Sint32                   error_number,
        mi::base::Message_severity   error_severity,
        std::string                  error_message,
        const INVALID_DOXYREFmi::IImpexp_state*     import_state) // not 0
    {
        std::ostringstream message;
        message << import_state->INVALID_DOXYREF()
                << ":" << import_state->INVALID_DOXYREF() << ": "
                << "Vanilla importer error " << error_number << ", "
                << "severity " << enum_to_str( error_severity) << ": "
                << error_message;
        // Report context of all parent import states from recursive
        // invocations of import_elements in their own lines with indentation.
        import_state = import_state->INVALID_DOXYREF();
        while( import_state) {
            message << "\n    included from: " << import_state->INVALID_DOXYREF()
                    << ":" << import_state->INVALID_DOXYREF();
            import_state = import_state->INVALID_DOXYREF();
        }
        result->INVALID_DOXYREF( error_number, error_severity, message.str().c_str());
        return result;
    }
};

// Returns true if the importer can handle the file type determined by the file name extension.
bool Vanilla_importer::test_file_type ( const char* extension ) const
{
    // This importer supports the file name extensions ".vnl" and ".van".
    mi::Size len = std::strlen( extension);
    return (len > 3)
        &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
          || ( 0 == strcmp( extension + len - 4, ".van")));
}

// Returns true if the importer can handle the file type determined by the file name extension
// and if the reader has sufficient capabilities for import.
bool Vanilla_importer::test_file_type( const char* extension,
                                       const INVALID_DOXYREFmi::IReader* reader ) const
{
    // Use magic header check if lookahead is available
    if ( reader->INVALID_DOXYREF()) {
        // File has to start with "VANILLA" and linebreak, which can
        // be \n or \r depending on the line ending convention in the file.
        const char** buffer = 0;
        mi::Sint64 n = reader->INVALID_DOXYREF( 8, buffer);
        return ( n >= 8) && (0 == std::strncmp( *buffer, "VANILLA", 7))
                         && ((*buffer[7] == '\n') || (*buffer[7] == '\r'));
    }
    // This importer supports the file name extensions ".vnl" and ".van".
    mi::Size len = std::strlen( extension);
    return (len > 3)
        &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
          || ( 0 == strcmp( extension + len - 4, ".van")));
}

// Imports all elements from the reader in a format determined by the file extension and
// (optionally) the lookahead of the reader.
INVALID_DOXYREFmi::IImport_result* Vanilla_importer::import_elements (
    mi::neuraylib::ITransaction* transaction,
    const char*                  extension,
    INVALID_DOXYREFmi::IReader*                 reader,
    const mi::IMap*              importer_options,
    INVALID_DOXYREFmi::IImpexp_state*           import_state) const
{
    // Create the importer result instance for the return value.
    // If that fails something is really wrong and we return NULL.
    INVALID_DOXYREFmi::IImport_result_ext* result
        = transaction->create<INVALID_DOXYREFmi::IImport_result_ext>( "Import_result_ext");
    if( !result)
        return NULL;

    // Get the 'prefix' option.
    MISTD::string prefix;
    if( importer_options && importer_options->has_key( "prefix")) {
        mi::base::Handle< const mi::IString> option(
            importer_options->get_value<mi::IString>( "prefix"));
        prefix = option->get_c_str();
    }

    // Get the 'list_elements' option.
    bool list_elements = false;
    if( importer_options && importer_options->has_key( "list_elements")) {
        mi::base::Handle< const mi::IBoolean> option(
            importer_options->get_value<mi::IBoolean>( "list_elements"));
        list_elements = option->get_value<bool>();
    }

    // Before we start parsing the file, we create a group that collects all
    // top-level elements. This will be our rootgroup.
    std::string root_group_name = prefix + "Vanilla::root_group";
    mi::base::Handle< mi::IGroup> rootgroup( transaction->create<INVALID_DOXYREFmi::IGroup>( "Group"));
    mi::Sint32 error_code = transaction->store( rootgroup.get(), root_group_name.c_str());
    if( error_code != 0)
        return report_error( transaction, result, 4010, mi::base::MESSAGE_SEVERITY_ERROR,
            "failed to create the root group", import_state);
    rootgroup = transaction->edit<INVALID_DOXYREFmi::IGroup>( root_group_name.c_str());

    // Register the rootgroup with the importer result.
    result->INVALID_DOXYREF( root_group_name.c_str());

    // If the element list flag is set, record the rootgroup element also in the
    // elements array of the result.
    if ( list_elements)
        result->INVALID_DOXYREF( root_group_name.c_str());

    // Assume it is a line based text format and read it line by line.
    // Assume lines are no longer than 256 chars, otherwise read it in pieces
    char buffer[257];
    while ( reader->INVALID_DOXYREF( buffer, 257) && buffer[0] != '\0') {

        // Periodically check whether the transaction is still open. If not, stop importing.
        if( !transaction->is_open())
            break;

        // Here you can process the buffer content of size len.
        // It corresponds to the line import_state->get_line_number() of the input file.
        mi::Size len = std::strlen( buffer);

        // We illustrate some actions triggered by fixed line numbers:
        // Line 3 of a ".vnl" file triggers the recursive inclusion of the file "test2.van" file
        // with a prefix.
        mi::Size ext_len = std::strlen( extension);
        if ( 3 == import_state->INVALID_DOXYREF()
             && (ext_len > 3)
             && 0 == strcmp( extension + ext_len - 4, ".vnl")) {
            // Get a IImport_api handle to call its import_elements() function
            mi::base::Handle< mi::neuraylib::IImport_api> import_api(
                m_iplugin_api->get_api_component<mi::neuraylib::IImport_api>());
            if ( ! import_api.is_valid_interface())
                // Error numbers from 4000 to 5999 are reserved for custom importer messages like
                // this one
                return report_error( transaction, result,
                   4001, mi::base::MESSAGE_SEVERITY_ERROR,
                   "did not get a valid IImport_api object, import failed", import_state);
            // Call the importer recursively to illustrate the handling of include files and
            // similar things. We trigger this import only on a ".vnl" file and include the fixed
            // file "test2.van". We give the included file an extra prefix "Prefix_".
            mi::base::Handle< mi::neuraylib::IFactory> factory(
                m_iplugin_api->get_api_component<mi::neuraylib::IFactory>());
            mi::base::Handle< mi::IString> child_prefix( factory->create<mi::IString>( "String"));
            child_prefix->set_c_str( "Prefix_");
            mi::base::Handle< mi::IMap> child_importer_options( factory->clone<mi::IMap>(
                importer_options));
            child_importer_options->erase( "prefix");
            child_importer_options->insert( "prefix", child_prefix.get());
            mi::base::Handle< const mi::IImport_result> include_result(
                import_api->import_elements( transaction, "test2.van", child_importer_options.get(),
                    import_state));
            // Safety check, if this fails, the import is not continued.
            if ( ! include_result.is_valid_interface())
                return report_error( transaction, result,
                   4002, mi::base::MESSAGE_SEVERITY_ERROR,
                   "import was not able to create result object, import failed", import_state);
            // Process the result. Even in the case of an error, we need to process the
            // elements array.
            if ( list_elements)
                result->INVALID_DOXYREF( include_result.get());
            // Report error of includes here as well.
            if ( include_result->get_errors_length() > 0) {
                // Append error messages of include to this result
                result->INVALID_DOXYREF( include_result.get());
                // Report the failure of the include as a separate message too
                report_error( transaction, result,
                    4003, mi::base::MESSAGE_SEVERITY_ERROR,
                    "including file 'test2.van' failed.", import_state);
            } else {
                // Recursive import was successful. The rootgroup of the
                // import is now appended to this rootgroup
                if ( 0 == include_result->get_rootgroup())
                    report_error( transaction, result,
                        4004, mi::base::MESSAGE_SEVERITY_ERROR,
                        "include file 'test2.van' did not contain a rootgroup", import_state);
                else
                    rootgroup->attach( include_result->get_rootgroup());
            }
        }

        // Line 4 triggers several error messages and adds an empty group to the rootgroup.
        if ( 4 == import_state->INVALID_DOXYREF()) {
            // Several error messages, file parsing continues
            report_error( transaction, result, 4005, mi::base::MESSAGE_SEVERITY_FATAL,
                          "test error in line 4", import_state);
            report_error( transaction, result, 4006, mi::base::MESSAGE_SEVERITY_ERROR,
                          "test error in line 4", import_state);
            report_error( transaction, result, 4007, mi::base::MESSAGE_SEVERITY_WARNING,
                          "test error in line 4", import_state);
            report_error( transaction, result, 4008, mi::base::MESSAGE_SEVERITY_INFO,
                          "test error in line 4", import_state);
            report_error( transaction, result, 4009, mi::base::MESSAGE_SEVERITY_VERBOSE,
                          "test error in line 4", import_state);
            // Create a group "Vanilla::Group1"
            std::string group_name = prefix + "Vanilla::Group1";
            mi::base::Handle< mi::IGroup> group( transaction->create<INVALID_DOXYREFmi::IGroup>( "Group"));
            mi::Sint32 error_code = transaction->store( group.get(), group_name.c_str());
            if( error_code != 0)
                report_error( transaction, result, 4011, mi::base::MESSAGE_SEVERITY_ERROR,
                    "unexpected error in line 4", import_state);
            else {
                // Add this group to the rootgroup
                rootgroup->attach( group_name.c_str());
                // If get_list_elements_flag is set, record the new element in the elements array
                // of the result.
                if ( list_elements)
                    result->INVALID_DOXYREF( group_name.c_str());
            }
        }

        // Handle line numbers, buffer might end in '\n' or not if line was too long.
        if ((len > 0) && ('\n' == buffer[len-1]))
            import_state->INVALID_DOXYREF();
    }
    if ( reader->INVALID_DOXYREF()) {
        // Normal end
        return result;
    }
    // Report error condition for a failed reader call
    return report_error( transaction,
                         result,
                         reader->INVALID_DOXYREF(),
                         mi::base::MESSAGE_SEVERITY_ERROR,
                         reader->INVALID_DOXYREF() ? reader->INVALID_DOXYREF() : "",
                         import_state);
}

‎/******************************************************************************
 * Copyright 1986, 2011 NVIDIA Corporation. All rights reserved.
 *****************************************************************************/

// examples/example_importer.cpp
//
// Demonstrates the implementation of custom importers

#include <iostream>

// Include code shared by all examples.
#include "example_shared.h"

// Include header file for the Vanilla importer.
#include "vanilla_importer.h"

// The importer.
mi::base::Handle< mi::neuraylib::IImporter> importer;

void configuration( mi::base::Handle< mi::neuraylib::INeuray> neuray)
{
    // Register the Vanilla importer.
    mi::base::Handle< mi::neuraylib::IExtension_api> extension_api(
        neuray->get_api_component<mi::neuraylib::IExtension_api>());
    check_success( extension_api.is_valid_interface());
    mi::base::Handle< mi::neuraylib::IPlugin_api> plugin_api(
        neuray->get_api_component<mi::neuraylib::IPlugin_api>());
    check_success( plugin_api.is_valid_interface());
    importer = new Vanilla_importer( plugin_api.get());
    check_success( extension_api->register_importer( importer.get()) == 0);
}
    
void test_importer( mi::base::Handle< mi::neuraylib::INeuray> neuray)
{
    // Get the database, the global scope, which is the root for all transactions,
    // and create a transaction.
    mi::base::Handle< mi::neuraylib::IDatabase> database(
        neuray->get_api_component<mi::neuraylib::IDatabase>());
    check_success( database.is_valid_interface());
    mi::base::Handle< mi::neuraylib::IScope> scope(
        database->get_global_scope());
    mi::base::Handle< mi::neuraylib::ITransaction> transaction(
        scope->create_transaction());
    check_success( transaction.is_valid_interface());

    // Prepare the importer options: 
    // We do not want to have an additional prefix, but a list of all imported elements.
    mi::base::Handle< mi::IBoolean> list_elements( transaction->create<mi::IBoolean>( "Boolean"));
    list_elements->set_value( true);
    mi::base::Handle< mi::IMap> importer_options( transaction->create<mi::IMap>( "Map<Interface>"));
    importer_options->insert( "list_elements", list_elements.get());

    // Import the file test1.vnl (implicitly using the Vanilla importer).
    mi::base::Handle< mi::neuraylib::IImport_api> import_api(
        neuray->get_api_component<mi::neuraylib::IImport_api>());
    check_success( import_api.is_valid_interface());
    mi::base::Handle< const mi::IImport_result> import_result(
        import_api->import_elements( transaction.get(), "test1.vnl", importer_options.get()));
    check_success( import_result.is_valid_interface());

    // Print all error messages
    for( mi::Size i = 0; i < import_result->get_errors_length(); ++i)
        std::cout << import_result->get_error_message( i) << std::endl;

    // Print all imported elements
    for( mi::Size i = 0; i < import_result->get_elements_length(); ++i)
        std::cout << import_result->get_element( i) << std::endl;
    
    // All transactions need to get committed or aborted, not really important in this example.
    transaction->commit();
}

int main( int argc, char* argv[])
{
    // Access the neuray library
    mi::base::Handle< mi::neuraylib::INeuray> neuray( load_and_get_ineuray());
    check_success( neuray.is_valid_interface());

    // Configure the neuray library
    configuration( neuray);

    // Start the neuray library
    check_success( neuray->start( true) == 0);

    // Test the Vanilla importer
    test_importer( neuray);

    // Shut down the neuray library
    check_success( neuray->shutdown() == 0);

    // Unregister the Vanilla importer.
    mi::base::Handle< mi::neuraylib::IExtension_api> extension_api(
        neuray->get_api_component<mi::neuraylib::IExtension_api>());
    check_success( extension_api->unregister_importer( importer.get()) == 0);
    importer = 0;
    extension_api = 0;
    neuray = 0;

    // Unload the neuray library
    check_success( unload());

    return EXIT_SUCCESS;
}