10
$\begingroup$

Suppose if there is a trained net:

SeedRandom[1234];
net = NetInitialize@NetChain[{5, 3}, "Input" -> 2];

It can predict image in mma:

net[{1, 2}]

{1.27735, -1.21455, -1.02647}

In C++,I want to use this net,so call wolframscript

wolframscript -code net=Import["... .wlnet"];net[{1,2}]

But whenever want to predict,I need to re-call the wolframscript,this way spend s much time(initializing environment,loading net...)

how to use wlnet in C++ fast and easy?

PS:In Matlab,I find it can create shared/dynamically linked library with mcc.

enter image description here

$\endgroup$
  • $\begingroup$ I'd reckon .wlnet is only intended as a format for use in Mathematica itself. Exporting your net to MXNet (which is what Mathematica's implementation of Neural Nets is based on) is a more feasible approach. There are two questions already about exporting to MXNet but from what I have taken from them currently the exporter is broken. $\endgroup$ – Sascha Jul 13 '17 at 6:55
  • $\begingroup$ Can you elaborate on what exactly you mean by "using" the network in C++? $\endgroup$ – Szabolcs Jul 13 '17 at 9:23
  • $\begingroup$ It means load the net and use it to predict on new data repeatedly.This process will insert in a Visual Studio project. $\endgroup$ – partida Jul 13 '17 at 10:34
  • $\begingroup$ Then you would probably want to use an existing deep learning framework. You need to choose one first (based on whatever criteria are relevant for your application), and then ask about how to export the network to that one. $\endgroup$ – Szabolcs Jul 13 '17 at 12:33
  • $\begingroup$ it seems Caffe supports C++ well.I am now seeking how to use this way now... $\endgroup$ – partida Jul 13 '17 at 12:38
6
$\begingroup$

In Mathematica 11.2, which will be released in the next month or so, it is as simple as writing Export["mynet.json", net, "MXNet"]. This will also produce a "params" file, and it is then straightforward to load both the JSON and params file from C++ using MXNet (see http://mxnet.io/api/python/symbol.html#mxnet.symbol.load).

$\endgroup$
  • $\begingroup$ But install MxNet using C++ interface is very difficult because of it need to be compiled and link library and head file.If only need to make a prediction using MxNet,can you provide some example about it. $\endgroup$ – partida Aug 31 '17 at 3:20
  • $\begingroup$ @partida: If you only need predictions, you can use the C predict API. Standalone header file, really easy to use. Here's a sample: github.com/apache/incubator-mxnet/blob/master/example/… $\endgroup$ – Niki Estner Sep 2 '17 at 9:45
  • $\begingroup$ @nikie thanks! I try $\endgroup$ – partida Sep 2 '17 at 10:36
  • 1
    $\begingroup$ @Taliesin Beynon Since 11.2 has released, could you post a example about How to use the .json and .param file in c++? $\endgroup$ – yode Oct 9 '17 at 9:28
  • 1
    $\begingroup$ @yode we don't have any material for that yet, but we hope to have some workflows and tutorials soon that will show how to do it with a worked example. $\endgroup$ – Taliesin Beynon Nov 3 '17 at 11:05
5
$\begingroup$

I will use two cases to make example:

First,Seq2Seq model

RandomSeed[1234];
net = NetInitialize@NetChain[{5, 3}, "Input" -> 2];
net[{1,2}]

{1.27735, -1.21455, -1.02647}

Export it(link How to export an MXNet?)

The model file can be download here. JSON and params

Note:Using Export["example.json",net,"MXNet"] will leads the output always be zeros in C++,I don't know why.

<< MXNetLink`;
<< NeuralNetworks`;
<< GeneralUtilities`;
net = NetInitialize@NetChain[{5, 3}, "Input" -> 2];

jsonPath = "example2.json";
Export[jsonPath, ToMXJSON[net][[1]], "String"]

paraPath = "example2.params";
f[str_] := 
 If[StringFreeQ[str, "Arrays"], str, 
  StringReplace[
   StringSplit[str, ".Arrays."] /. {a_, b_} :> 
     StringJoin[{"arg:", a, "_", b}], {"Weights" -> "weight", 
    "Biases" -> "bias"}]]

plan = ToMXPlan[net];
NDArrayExport[paraPath, NDArrayCreate /@ KeyMap[f, plan["ArgumentArrays"]]]

you will get JSON and params file.But the name of weights in JSON and params are different.You sould modify the json file like this

enter image description here

Then go to C++,I am using VS2015. First,you should make a new project.Then copy the libmxnet.dll and libmxnet.lib to workspace(from where mxnet installed,you can search these two files),and download the head file:c_predict_api.h.

Maybe you need some DLL. such like this

enter image description here

The usage of MXNet API can find c_predict_api.h File Reference

Maybe the input_shape_indptr is difficult to understand, you can refer my question

Maybe you want to predict a mini-batch in C++,refer my answer

The full code to make prediction:

#include <stdio.h>

// Path for c_predict_api
#include <mxnet/c_predict_api.h>

#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include <assert.h>

// Read file to buffer
class BufferFile {
public:
    std::string file_path_;
    int length_;
    char* buffer_;

    explicit BufferFile(std::string file_path)
        :file_path_(file_path) {

        std::ifstream ifs(file_path.c_str(), std::ios::in | std::ios::binary);
        if (!ifs) {
            std::cerr << "Can't open the file. Please check " << file_path << ". \n";
            length_ = 0;
            buffer_ = NULL;
            return;
        }

        ifs.seekg(0, std::ios::end);
        length_ = ifs.tellg();
        ifs.seekg(0, std::ios::beg);
        std::cout << file_path.c_str() << " ... " << length_ << " bytes\n";

        buffer_ = new char[sizeof(char) * length_];
        ifs.read(buffer_, length_);
        ifs.close();
    }

    int GetLength() {
        return length_;
    }
    char* GetBuffer() {
        return buffer_;
    }

    ~BufferFile() {
        if (buffer_) {
            delete[] buffer_;
            buffer_ = NULL;
        }
    }
};

void PrintOutputResult(const std::vector<float>& data) {
    for (int i = 0; i < static_cast<int>(data.size()); i++) {
        printf("%.8f\n", data[i]);
    }
    printf("\n");
}

int main(int argc, char* argv[]) {

    // Models path for your model, you have to modify it
    std::string json_file = "./simple prediction model/example2.json";
    std::string param_file = "./simple prediction model/example2.params";

    BufferFile json_data(json_file);
    BufferFile param_data(param_file);

    // Parameters
    int dev_type = 1;  // 1: cpu, 2: gpu
    int dev_id = 1;  // arbitrary.
    mx_uint num_input_nodes = 1;  // 1 for feedforward
    const char* input_key[1] = { "Input" };
    const char** input_keys = input_key;

    // input-dims
    int data_len = 2;

    const mx_uint input_shape_indptr[2] = { 0, 2 };
    const mx_uint input_shape_data[2] = { 1,static_cast<mx_uint>(data_len) };
    PredictorHandle pred_hnd = 0;

    if (json_data.GetLength() == 0 || param_data.GetLength() == 0)
        return -1;

    // Create Predictor
    assert(0==MXPredCreate((const char*)json_data.GetBuffer(),
        (const char*)param_data.GetBuffer(),
        static_cast<size_t>(param_data.GetLength()),
        dev_type,
        dev_id,
        num_input_nodes,
        input_keys,
        input_shape_indptr,
        input_shape_data,
        &pred_hnd));
    assert(pred_hnd);

    int input_data_size = 2;

    std::vector<mx_float> vector_data = std::vector<mx_float>(input_data_size);
    vector_data[0] = 1;
    vector_data[1] = 2;

    MXPredSetInput(pred_hnd, "Input", vector_data.data(), 2);

    // Do Predict Forward
    MXPredForward(pred_hnd);

    mx_uint output_index = 0;

    mx_uint *shape = 0;
    //shape相当于1*3的向量
    mx_uint shape_len;

    // Get Output Result
    MXPredGetOutputShape(pred_hnd, output_index, &shape, &shape_len);

    size_t size = 1;
    for (mx_uint i = 0; i < shape_len; ++i) size *= shape[i];

    std::vector<float> data(size);

    assert(0==MXPredGetOutput(pred_hnd, output_index, &(data[0]), size));

    // Release Predictor
    MXPredFree(pred_hnd);

    // Print Output Data
    PrintOutputResult(data);
    return 0;
}

enter image description here

Second: image classification link image-classification-predict.cc

You should have install opencv so that can read images. I haven't test the net from wolfram,the model from MXNet example. This is mean file and Category Table of classes

The full code to make prediction:

#include <stdio.h>

// Path for c_predict_api
#include <mxnet/c_predict_api.h>

#include <iostream>
#include <fstream>
#include <string>
#include <vector>

#include <opencv2/opencv.hpp>

const mx_float DEFAULT_MEAN = 117.0;

// Read file to buffer
class BufferFile {
 public :
    std::string file_path_;
    int length_;
    char* buffer_;

    explicit BufferFile(std::string file_path)
    :file_path_(file_path) {

        std::ifstream ifs(file_path.c_str(), std::ios::in | std::ios::binary);
        if (!ifs) {
            std::cerr << "Can't open the file. Please check " << file_path << ". \n";
            length_ = 0;
            buffer_ = NULL;
            return;
        }

        ifs.seekg(0, std::ios::end);
        length_ = ifs.tellg();
        ifs.seekg(0, std::ios::beg);
        std::cout << file_path.c_str() << " ... "<< length_ << " bytes\n";

        buffer_ = new char[sizeof(char) * length_];
        ifs.read(buffer_, length_);
        ifs.close();
    }

    int GetLength() {
        return length_;
    }
    char* GetBuffer() {
        return buffer_;
    }

    ~BufferFile() {
        if (buffer_) {
          delete[] buffer_;
          buffer_ = NULL;
        }
    }
};

void GetImageFile(const std::string image_file,
                  mx_float* image_data, const int channels,
                  const cv::Size resize_size, const mx_float* mean_data = nullptr) {
    // Read all kinds of file into a BGR color 3 channels image
    cv::Mat im_ori = cv::imread(image_file, cv::IMREAD_COLOR);

    if (im_ori.empty()) {
        std::cerr << "Can't open the image. Please check " << image_file << ". \n";
        assert(false);
    }
    else
        std::cout << image_file.c_str() << " bytes\n";

    cv::Mat im;

    resize(im_ori, im, resize_size);

    int size = im.rows * im.cols * channels;

    mx_float* ptr_image_r = image_data;
    mx_float* ptr_image_g = image_data + size / 3;
    mx_float* ptr_image_b = image_data + size / 3 * 2;

    float mean_b, mean_g, mean_r;
    mean_b = mean_g = mean_r = DEFAULT_MEAN;

    for (int i = 0; i < im.rows; i++) {
        uchar* data = im.ptr<uchar>(i);

        for (int j = 0; j < im.cols; j++) {
            if (mean_data) {
                mean_r = *mean_data;
                if (channels > 1) {
                    mean_g = *(mean_data + size / 3);
                    mean_b = *(mean_data + size / 3 * 2);
                }
               mean_data++;
            }
            if (channels > 1) {
                *ptr_image_g++ = static_cast<mx_float>(*data++) - mean_g;
                *ptr_image_b++ = static_cast<mx_float>(*data++) - mean_b;
            }

            *ptr_image_r++ = static_cast<mx_float>(*data++) - mean_r;;
        }
    }
}

// LoadSynsets
// Code from : https://github.com/pertusa/mxnet_predict_cc/blob/master/mxnet_predict.cc
std::vector<std::string> LoadSynset(std::string synset_file) {
    std::ifstream fi(synset_file.c_str());

    if ( !fi.is_open() ) {
        std::cerr << "Error opening synset file " << synset_file << std::endl;
        assert(false);
    }

    std::vector<std::string> output;

    std::string synset, lemma;
    while ( fi >> synset ) {
        getline(fi, lemma);
        output.push_back(lemma);
    }

    fi.close();

    return output;
}

void PrintOutputResult(const std::vector<float>& data, const std::vector<std::string>& synset) {
    if (data.size() != synset.size()) {
        std::cerr << "Result data and synset size does not match!" << std::endl;
    }

    float best_accuracy = 0.0;
    int best_idx = 0;

    for ( int i = 0; i < static_cast<int>(data.size()); i++ ) {
        printf("Accuracy[%d] = %.8f\n", i, data[i]);

        if ( data[i] > best_accuracy ) {
            best_accuracy = data[i];
            best_idx = i;
        }
    }

    printf("Best Result: [%s] id = %d, accuracy = %.8f\n",
    synset[best_idx].c_str(), best_idx, best_accuracy);
}

int main(int argc, char* argv[]) {
    if (argc < 2) {
        std::cout << "No test image here." << std::endl
        << "Usage: ./image-classification-predict apple.jpg" << std::endl;
        return 0;
    }

    std::string test_file;
    test_file = std::string(argv[1]);

    // Models path for your model, you have to modify it
    std::string json_file = "./Inception model/Inception-BN-symbol.json";
    std::string param_file = "./Inception model/Inception-BN-0126.params";
    std::string synset_file = "./Inception model/synset.txt";
    std::string nd_file = "./Inception model/mean_224.nd";

    BufferFile json_data(json_file);
    BufferFile param_data(param_file);

    // Parameters
    int dev_type = 1;  // 1: cpu, 2: gpu
    int dev_id = 0;  // arbitrary.
    mx_uint num_input_nodes = 1;  // 1 for feedforward
    const char* input_key[1] = {"data"};
    const char** input_keys = input_key;

    // Image size and channels
    int width = 224;
    int height = 224;
    int channels = 3;

    const mx_uint input_shape_indptr[2] = { 0, 4 };
    const mx_uint input_shape_data[4] = { 1,
                                        static_cast<mx_uint>(channels),
                                        static_cast<mx_uint>(height),
                                        static_cast<mx_uint>(width)};
    PredictorHandle pred_hnd = 0;

    if (json_data.GetLength() == 0 ||
        param_data.GetLength() == 0) {
        return -1;
    }

    // Create Predictor
    MXPredCreate((const char*)json_data.GetBuffer(),
                 (const char*)param_data.GetBuffer(),
                 static_cast<size_t>(param_data.GetLength()),
                 dev_type,
                 dev_id,
                 num_input_nodes,
                 input_keys,
                 input_shape_indptr,
                 input_shape_data,
                 &pred_hnd);
    assert(pred_hnd);

    int image_size = width * height * channels;

    // Read Mean Data
    const mx_float* nd_data = NULL;
    NDListHandle nd_hnd = 0;
    BufferFile nd_buf(nd_file);

    if (nd_buf.GetLength() > 0) {
        mx_uint nd_index = 0;
        mx_uint nd_len;
        const mx_uint* nd_shape = 0;
        const char* nd_key = 0;
        mx_uint nd_ndim = 0;

        MXNDListCreate((const char*)nd_buf.GetBuffer(),
                   nd_buf.GetLength(),
                   &nd_hnd, &nd_len);

        MXNDListGet(nd_hnd, nd_index, &nd_key, &nd_data, &nd_shape, &nd_ndim);
    }

    // Read Image Data
    std::vector<mx_float> image_data = std::vector<mx_float>(image_size);

    GetImageFile(test_file, image_data.data(),
                 channels, cv::Size(width, height), nd_data);

    // Set Input Image
    MXPredSetInput(pred_hnd, "data", image_data.data(), image_size);

    // Do Predict Forward
    MXPredForward(pred_hnd);

    mx_uint output_index = 0;

    mx_uint *shape = 0;
    mx_uint shape_len;

    // Get Output Result
    MXPredGetOutputShape(pred_hnd, output_index, &shape, &shape_len);

    size_t size = 1;
    for (mx_uint i = 0; i < shape_len; ++i) size *= shape[i];

    std::vector<float> data(size);

    MXPredGetOutput(pred_hnd, output_index, &(data[0]), size);

    // Release NDList
    if (nd_hnd)
      MXNDListFree(nd_hnd);

    // Release Predictor
    MXPredFree(pred_hnd);

    // Synset path for your model, you have to modify it
    std::vector<std::string> synset = LoadSynset(synset_file);

    // Print Output Data
    PrintOutputResult(data, synset);
    return 0;
}

enter image description here

link How to delete the parameter of a Neural Network?,the result is different,I think maybe it doesn't have mean file,but stage is acceptable,is it OK?

enter image description here

$\endgroup$
  • $\begingroup$ There is a built-in NeuralNetworks`MXNetExport,I don't sure it can help or not. $\endgroup$ – yode Oct 11 '17 at 6:55
1
$\begingroup$

The right fix for the params file is this, it is fully exaplained in https://mathematica.stackexchange.com/a/162479/54287. The params file obtained by Mathematica misses of the string "arg" at the front of each key only but does not need to modify the strings in {"Weights" -> "weight", "Biases" -> "bias"}.

In the following the Mathematica code:

(*Loading the needed packages*)
<< MXNetLink`;
<< NeuralNetworks`;
<< GeneralUtilities`;
(*The network name*)
networkStr = { "MathematicaNet"};
(*Is the number of training epochs, it is possible to load the model \
at different training epochs number*)
numberOfEpochs = 0;

(*THE FIX*)
FixMXNnet[str_] := StringJoin[{"arg:", str}]


Export[StringJoin[{networkStr[[1]], "-symbol.json"}], 
 ToMXJSON[net][[1]], "String"]
plan = ToMXPlan[net];
NDArrayExport[
 StringJoin[{networkStr[[1]], "-", 
   StringPadLeft[ToString[numberOfEpochs], 4, "0"], ".params"}], 
 NDArrayCreate /@ KeyMap[FixMXNnet, plan["ArgumentArrays"]]]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.