Tuesday, August 23, 2011

The maximum number of matrix dimensions in MATLAB

[ Background: I was asked what the maximum number of matrix dimensions was in MATLAB today. I responded as follows. ]

You are only limited by the amount of memory available and the maximum number of ELEMENTS (as opposed to dimensions) in a matrix. The actual number of dimensions is just a detail about how the memory is indexed. You can reshape any existing matrix to any number of dimensions (I'll give details below). You can only create a new matrix if it abides by the memory and element limits that vary by computer.

To find out the maximum number of elements for a matrix on your computer, use the MATLAB command "computer" (do "help computer" for details). For example:
[~,maxsize,~]=computer
tells me that I can have 2.8147e+14 elements in matrices on my computer. So I better be sure that:
(number of rows)
   × (number of columns)
   × (number of cubes)
   × (number of 4-th dimensional thinggies)
   × (...)
is less than that number.

To find out about memory limits on your system see, the command "memory" ("help memory" or "doc memory"). Unfortunately, memory may not be available on your system. Alternatively, you can see:

http://www.mathworks.com/support/tech-notes/1100/1110.html

for information about memory limits in MATLAB. For information about the maximum number of elements (and the command "computer" that I discussed above), see (UPDATE: MATLAB has moved this page, and this link doesn't land in the right spot anymore):

http://www.mathworks.com/support/tech-notes/1200/1207.html#15

Regarding dimensions, you can use the command "reshape" to re-index any existing matrix. For example, if I start with the column vector:
A=ones(100,1)
I can turn it into a row vector:
newA = reshape(A, 1, 100)
or a matrix of any number of dimensions so long as the number of elements is still 100.
newA = reshape( A, 2, 2, 25 )
newA = reshape( A, 1, 1, 1, 1, 1, 1, 1, 1, 1, 100, 1 )
newA = reshape( A, 1, 1, 1, 2, 1, 50, 1, 1, 1, 1, 1, 1, 1, 1 )
% etc.
Now, I'm assuming you're using regular MATLAB matrices. Alternatively, you can use sparse matrices so long as you limit yourself to functions that work with sparse matrices:
help sparfun
A sparse matrix stores an index with every element. That lets it "skip over" the 0 elements of the matrix. Consequently, you can store VERY large matrices with an abstract number of elements far larger than anything you can work with in MATLAB... however, most of those abstract elements will be 0.

2 comments:

Anonymous said...

Wouldn't it be the maximum value of an unsigned long integer that was defined when Matlab was compiled? Or does Matlab have its own internal arbitrary digit-length math library(ies)? In either case, why would the amount of system memory define maximal size of matrices? The heap can extended in a number of ways on a single client and even more-so in an MPI environment. Perhaps you were simply trying to explain Matlab's storage method (indexed number store) versus its language-level representation.

Ted Pavlic said...

Anonymous commenter -- I don't understand your confusion. I've also linked to pages at Mathworks that provide more information. Take, for example, the information at this link (which is the first link I give in the above):

MATLAB Central: What is the maximum matrix size for each platform?

The accepted answer includes this paragraph, which should alleviate any confusion you have (note the text "RAM plus swap file"):

"The largest matrix or array that can be created in MATLAB, together with the total workspace size (sum of all variables), on each platform is given in the table and graph below. It assumes that MATLAB has just been launched with no major processing carried out in a startup.m file and that there is unlimited system memory (RAM plus swap file) available. These numbers will be less if there is insufficient system memory available, usually due to the swap file being too small."