Old Format

The CDF-1 file format is the default file format. The definitive resource for these limits can be found at Unidata's NetCDF Classic Format Limitations page

Extending Limits

The relatively recent CDF-2 file format relaxes many of the above limitations. In order to create a CDF-2 formatted file, you must pass the flag NC_64BIT_OFFSET when creating the dataset. Parallel-NetCDF supports the CDF-2 file format as well, though shares the same limits as serial NetCDF. Read more at Unidata's 64-bit Offset Format Limitations page.

Record vs. Non-Record

One subtle point about the CDF file format limitations is that record variables (those with one NC_UNLIMITED dimension) can be quite large. The product of the fixed-sized dimensions of a record variable defines the size of a single record. There can be quite a few of these records in a record variable, though, so that can be one way around the file format limitations on record size.

The drawback of using record variables is that with multiple record variables, the data for each variable are interleaved on a per-record basis. This data layout makes it much harder to figure out the optimal access pattern. Ideally, though, programs would read and write record variables one record at a time.

Work in progress: Relaxing limits even further

Argonne and Northwestern are working together to define a CDF-3 file format. More information at the NewFileFormat page