Skip to content

Saving 3D array into .npy file for large dimension gets corrupted output #100

Description

@sahilhassanzee

I'm trying to put a 3D array of dimension (10000, 160, 346) in an .npy file using the append mode as shown in the code below:

#include "cnpy.h"
#include <vector>
#include <cstdlib>
constexpr int N = 346*160;

int main(int argc, char* argv[]) {
    int ROWS = 10000;

    // Create a vector
    std::vector<double> vec (N,0);

    // Fill the vector with data
    for(int i=0; i < N; i++) {
        vec[i] = i;
    }

    // Append the vector ROWS number of times into an npz file
    for (int t=0; t<ROWS; t++){
        cnpy::npy_save("sample.npy", &vec[0], {1,160,346}, "a");
    }

    return 0;
}

If I now try to open this file using the following Python code:

import numpy as np
x = np.load('sample.npy')

I get the following error:

ValueError: cannot reshape array of size 553599998 into shape (10000,160,346)

So, the size of array in the saved sample.npy is 2 Bytes smaller than what's expected. I have observed arrays upto 16 Bytes off from the expected size for larger size. Could you help resolve this issue?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions