>Can you give me some more advice on how to allow the uploading of large
>(< 17 MB) files.
>
>We are using the "older (non-database-oriented) scripts". When etd1.pl
>is called it uses CGI.pm to put the contents of a submission form into a
>variable ($query). Through the prodigious use of the CGI::Carp function,
>I was able to learn that an 18 MB kills the script with an out of memory
>error at the following line:
>
>  $query = new CGI;
>
>My hardware has gobbs of RAM. What sorts of alternatives are suggested
>for eliminating this kind of problem?
>
>--
>Eric Lease Morgan
>Digital Library Initiatives Department, NCSU Libraries
>http://www.lib.ncsu.edu/staff/morgan/

Hello:

Unfortunately, the fact the the script dies when it first tries to allocate
memory for the CGI data is not the best sign, in that you probably won't be
able to improve the memory usage by changing the script itself, short of
completely abandoning CGI.pm for your file upload scripts.

I did a little experiment on our equipment a while back, and watched the
memory usage (with "top") as the file upload script tried to process a
large file.  Try this with a file smaller than the limit, then see how much
real memory is being used over the course of the upload.

Also, I've attached a script that's designed to overload perl's memory,
it's useful in that you get a real-world estimate (within 1 Mb) of the
total amount of memory perl is going to be willing to use.  I've found this
to be less than or equal to half the free memory on a given system.  Find
out both of these things, the total amount of memory perl is willing to
use, and the relationship between the size of the file and the amount of
memory the script uses, it may prove enlightening.

Also, check to be absolutely sure that both the web server and perl have
enough space free to write their temporary versions of the file being
uploaded.  Check the docs for both for the location of the temporary files
they use.

I don't know if any of this will be useful, but it should at least let you
examine your problem further.  It may be that you'll have to settle (for
now) for an 18Mb limit on the file upload size and arrange for ftp uploads
for individual files larger than 18Mb.  In our case, once the file size
limit climbed to about 23 Mb (following our last upgrade), we found that
only a few authors each semester required the FTP account.   That having
been said, as long as you're willing to keep working on this, keep me
informed and I'll provide what help I can.

Regards,



Anthony Atkins


========================================================
Anthony Atkins, Technical Director         [log in to unmask]
  Scholarly Communications Project                  (540)231-9235
  Newman Library, VA Tech                   http://scholar.lib.vt.edu
========================================================