Strings in Delphi should be able to store 2GB, right?
I got an exception after 4MB.
I have to create a report about 800-1000 files in a folder.
I've tried to write the report in a TRichEdit component but it was T SLW because the refresh
after each line I added. BeginUpdate didn't help at all. Actually it has no effect.
So I decided to write it directly on the HDD as a HTML file.
I made a new form and a global variable (called Raport) where I store the HTML:
var Raport: string;
procedure TFrmHTML.AddLine(s: String);
begin
Raport:= Raport+ CRLF+ '<br>'+ s;
end;
At the end of the program I discharge the content of that variable in a file.
The problem is that after exactly 4MB it gives me an EMemory error.
It appears when I try to add more characters to that Raport variable.
The CPU window shows me that the exception was 'born' in the @StrSetLength function which is not
documented in my Help file (Delphi 7).
After this the program quit the main loop (leaving few file not processed).
What it is strange it the fact that after the exception is raised and closed, I continue to write
to that variable (the end of the HTML </body</html>) and it accept without rising the exception.
Finally the string is saved gracefully on the HDD.
I have 768 BM of RAM in my computer. All the time AT LEAST 250MB are free (I use CacheMan XP to
monitor my memory).
hour ago the program worked ok, till the end (it processed all 822 files).
The memory requirements were:
After it finished: 37MB
Peak memory : 39MB
Virtual mem : 32MB
In my system the page file usage was 565MB (I had other huge applications running).
Why it runs out of memory? It isn't supposed to use VM when it runs out of real memory (which it
isn't my case anyway).
What will happen if I want to process more that 1000 files?
PS: I don't want to use a third party Report component. I've took a look at few free and trial
components.
They where to expensive, or to slow, or much to complex (and in the same time big in size).
I just want to write line of text with monospaced characters/font and some letters should be
colored in red. That's all.
and the traveler died, stroked by the beauty of the landscape.
THE MRNING F THE MAGICIANS
Louis Pawels & Jacques Bergier
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Delphi-Talk mailing list -Delphi-Talk (AT) elists (DOT) org

Hello Human,

You've run into memory fragmentation. I would sugest you to use NexusMM
($120) or FastMM (free). Sorry, I cannot remember the URLs. Google is your
friend.

Regards,

SZ

Message
From: "Human" <newfedra (AT) yahoo (DOT) com>
To: "Delphi-Talk Discussion List" <delphi-talk (AT) elists (DOT) org>
Sent: Friday, March 03, 2006 1:50 PM
Subject: Strings in Delphi should be 2GB, right?

Strings in Delphi should be able to store 2GB, right?
I got an exception after 4MB.
>
>
>
I have to create a report about 800-1000 files in a folder.
I've tried to write the report in a TRichEdit component but it was T SLW
because the refresh
after each line I added. BeginUpdate didn't help at all. Actually it has
no effect.

So I decided to write it directly on the HDD as a HTML file.
I made a new form and a global variable (called Raport) where I store the
HTML:

var Raport: string;
procedure TFrmHTML.AddLine(s: String);
begin
Raport:= Raport+ CRLF+ '<br>'+ s;
end;

At the end of the program I discharge the content of that variable in a
file.
The problem is that after exactly 4MB it gives me an EMemory error.
It appears when I try to add more characters to that Raport variable.
The CPU window shows me that the exception was 'born' in the @StrSetLength
function which is not
documented in my Help file (Delphi 7).
After this the program quit the main loop (leaving few file not
processed).

What it is strange it the fact that after the exception is raised and
closed, I continue to write
to that variable (the end of the HTML </body</html>) and it accept
without rising the exception.
Finally the string is saved gracefully on the HDD.

I have 768 BM of RAM in my computer. All the time AT LEAST 250MB are free
(I use CacheMan XP to
monitor my memory).
hour ago the program worked ok, till the end (it processed all 822
files).

The memory requirements were:
After it finished: 37MB
Peak memory : 39MB
Virtual mem : 32MB
In my system the page file usage was 565MB (I had other huge applications
running).
--
Why it runs out of memory? It isn't supposed to use VM when it runs out of
real memory (which it
isn't my case anyway).
What will happen if I want to process more that 1000 files?
--
PS: I don't want to use a third party Report component. I've took a look
at few free and trial
components.
They where to expensive, or to slow, or much to complex (and in the same
time big in size).
I just want to write line of text with monospaced characters/font and some
letters should be
colored in red. That's all.
--
and the traveler died, stroked by the beauty of the landscape.

THE MRNING F THE MAGICIANS
Louis Pawels & Jacques Bergier

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

Delphi-Talk mailing list -Delphi-Talk (AT) elists (DOT) org

Delphi-Talk mailing list -Delphi-Talk (AT) elists (DOT) org

Hello Human,

Anyway, I'm still looking for a much simpler solution,
for example a function that can build long
strings without fragmenting the memory.
My string will not go above 10MB which I think is not so big.

It is not difficult. Allocate a buffer that is large enough to hold the
maximum size you presume. You write to it using Move procedure and
increment an offset variable that points to the end of the write.

If you need more room that needed in the buffer, just set a flag en
allocate a second buffer.

When the job is dont, then dont free the buffers. Just re-use the
allocated memory for the second (and next) go's.

This way you only allocate memory 1 time and you will not have memory
fragmentation.

Another option is to write direcly to disk instead of building it first
in memory. But probably you have your own reasons for it.

Rgds, Wilfried
http://www.mestdagh.biz

Delphi-Talk mailing list -Delphi-Talk (AT) elists (DOT) org

Hello Human,

Also, I discovered that I need to keep the whole report in memory
until it is fully built, then I have to sort the paragraphs in a
special order. So, writing to disk is excluded.

Then you have to do a lot of Move's after the whole buffer is build. To
avoid that you can allocate a buffer per paragraph and add it to a
TList. Make each buffer the maximum predicted size of the paragraph.

For the sorting you just have to move pointers in the list. Very quick.

Rgds, Wilfried
http://www.mestdagh.biz

Delphi-Talk mailing list -Delphi-Talk (AT) elists (DOT) org

Hello Human,

Im a little bit disappointed. I never imagine it will be so
complicated to write a 1-10MB string.

This is difficult to optimize. Concatenating strings is in fact ar
reallocation of memory. the larger it is the more difficult to find a
contiginous block of memory. Suppose at a certain moment you have 100 KB
allocated. Then you add 10 KB. What happens is a new block of 110 KB is
allocated, then the both are copied into it, then the first 100 KB is
released. So you need at least double memory. Meanwhile it is not sure
the first is released by the system immediatly. At a certain moment your
memory is so fragmented that there are no large blocks anymore.
parts of your application use the heap as whell.

Solution is to use and re-use over and over again same allocated block.
Also use stack space when ever possible. It cost no cpu time and no
memory allocation / deallocation.

Rgds, Wilfried
http://www.mestdagh.biz

Delphi-Talk mailing list -Delphi-Talk (AT) elists (DOT) org