NAVIGATION
CATEGORIES
REFERRENCE
LINKS
OutOfMemoryException Processing Velocity Template
19 answers - 6051 bytes -
Well, a file, use a FileWriter.
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:48 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
I printed out the available memory before and after I call
"mergeTemplate":
Free memory before merge: 217393352
Free memory after merge: 16271392
so I would assume the StringWriter is approximately 200MB after the
merge. What should I use/do instead?
-Lawrence
Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
Why do you use StringWriter? That is cached in the memory.
How big is the StringWriter after the merge?
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:22 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
Alright, I ran this little piece of code:
public static void main( String args[] ) throws Exception
{
/* first, we init the runtime engine. Defaults are fine. */
Velocity.init();
/* lets make a Context and put data into it */
VelocityContext context = new VelocityContext();
List results = ReportBean.runLargeReport();
context.put("List", results);
/* lets render a template */
StringWriter w = new StringWriter();
Velocity.mergeTemplate("/myfile.vm", context, w );
System.out.println(" template : " + w );
/* lets make our own string to render */
}
results is a List of 66000 objects and requires about 8megs of memory.
The "mergeTemplate" requires at least 230 MB of memory. Any
suggestions?
-L
Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
Hi,
I guess the writer Ilkka talked about is when using Velocity as a
standalone template engine.
Right now I don't have a good clue, could you run Velocity as a
standalone template engine and render the page to see if there is a
huge memory usage?
Cheers,
Jian
5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
Sorry, I'm not sure what you mean by what "kind of writer" I'm
using. Where would I find out? How would I change this (to maybe a
StringBuffer writer?)
Also, each column contains a String that is at most, 200
characters,
most are closer to 10-20. There are a total of 10 columns.
Thanks.
-Lawrence
Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
What kind of writer are you using and how long are the items? 66
000 HTML
table rows with 5 numeric columns consumes about 32 MB when
buffered into a
string writer. If the items are longer, it takes much more
memory.
-- Ilkka
--
lawrence (AT) marchex (DOT) com wrote:
>I tried putting static HTML in the #foreach loop, same problem.
Then I
>tried setting the large list as a "local" varible:
>
>#set ($localList = $List)
>#foreach($row in $localList)
>
>
>but that didnt help either. I don't see why velocity should
use
250+ MB
>of memory iterating over a list ~8MB in size
>-L
>
Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
>
>>Well, I don't have a good clue now.
>>
>>What about do the debugging step by step, i.e., use just
static
html
>>in your #foreach loop? Will that blow up the memory?
>>
>>Cheers,
>>
>>Jian
>>
>>
>>
>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
>>
Thanks for the suggestion. I tried putting that, along with
file.resource.loader.cache = true
in my velocity.properties file. Both did not work. Any
other
suggestions?
Thanks!
L
Thu, May 05, 2005 at 02:11:28PM -0700, jian chen wrote:
Hi,
This maybe because you haven't turned on template caching. I
had
similar problem before, and even if I only used
evaluate()
method
and load the template using my own java program, it happened
when the
for loop is lot of iterations.
The bottom line for my problem is I need to turn on template
caching,
so, velocity will not try to cache the objects it parsed
during each
time of the iteration.
I did:
ri.setProperty(,
"true");
where ri is a runtime instance.
Will this fix your issue?
Cheers,
Jian
5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
wrote:
I am getting an MemoryException:
Invocation of
method 'get' in class
threw exception class MemoryError : null
when I am creating a large velocity file. The vm file
itself
is
fairly small, but this piece of velocity is causing the
exception:
foreach($row in $List)
<tr>
<td height="20" class="tbltxt">$!row.item1</td>
<td class="tbltxt">$!row.item2</td>
<td class="tbltxt">$!row.item3</td>
<td class="tbltxt">$!row.item4</td>
<td class="tbltxt">$!number.format('currency',
$!row.item5)</td>
</tr>
end
List is a list of relatively small objects (5-6 Strings, a
couple of
BigDecimals), but the List itself contains approximately
66,000
entries. Unfortunately, unless I set the tomcat JVM to use
a
heap
size of at least 256M, this velocity template throws an
MemoryException when it is being processed.
Any suggestions?
Thanks.
L
--No.1 | | 6474 bytes |
| 
Alright, I changed the Writer to be:
StreamWriter w = new StreamWriter(new FStream("test.html"));
And this fixes the memory problem.
However, my application is running Velocity as a servlet, how should I extend the VelocityServlet to use an StreamWriter intead of a StringWriter? Should I post a different email thread about this?
-L
Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
Well, a file, use a FileWriter.
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:48 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
I printed out the available memory before and after I call
"mergeTemplate":
Free memory before merge: 217393352
Free memory after merge: 16271392
so I would assume the StringWriter is approximately 200MB after the
merge. What should I use/do instead?
-Lawrence
Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
Why do you use StringWriter? That is cached in the memory.
How big is the StringWriter after the merge?
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:22 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
Alright, I ran this little piece of code:
public static void main( String args[] ) throws Exception
{
/* first, we init the runtime engine. Defaults are fine. */
Velocity.init();
/* lets make a Context and put data into it */
VelocityContext context = new VelocityContext();
List results = ReportBean.runLargeReport();
context.put("List", results);
/* lets render a template */
StringWriter w = new StringWriter();
Velocity.mergeTemplate("/myfile.vm", context, w );
System.out.println(" template : " + w );
/* lets make our own string to render */
}
results is a List of 66000 objects and requires about 8megs of memory.
The "mergeTemplate" requires at least 230 MB of memory. Any
suggestions?
-L
Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
Hi,
I guess the writer Ilkka talked about is when using Velocity as a
standalone template engine.
Right now I don't have a good clue, could you run Velocity as a
standalone template engine and render the page to see if there is a
huge memory usage?
Cheers,
Jian
5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
Sorry, I'm not sure what you mean by what "kind of writer" I'm
using. Where would I find out? How would I change this (to maybe a
StringBuffer writer?)
Also, each column contains a String that is at most, 200
characters,
most are closer to 10-20. There are a total of 10 columns.
Thanks.
-Lawrence
Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
What kind of writer are you using and how long are the items? 66
000 HTML
table rows with 5 numeric columns consumes about 32 MB when
buffered into a
string writer. If the items are longer, it takes much more
memory.
-- Ilkka
--
lawrence (AT) marchex (DOT) com wrote:
>I tried putting static HTML in the #foreach loop, same problem.
Then I
>tried setting the large list as a "local" varible:
>
>#set ($localList = $List)
>#foreach($row in $localList)
>
>
>but that didnt help either. I don't see why velocity should
use
250+ MB
>of memory iterating over a list ~8MB in size
>-L
>
Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
>
>>Well, I don't have a good clue now.
>>
>>What about do the debugging step by step, i.e., use just
static
html
>>in your #foreach loop? Will that blow up the memory?
>>
>>Cheers,
>>
>>Jian
>>
>>
>>
>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
>>
Thanks for the suggestion. I tried putting that, along with
file.resource.loader.cache = true
in my velocity.properties file. Both did not work. Any
other
suggestions?
Thanks!
L
Thu, May 05, 2005 at 02:11:28PM -0700, jian chen wrote:
Hi,
This maybe because you haven't turned on template caching. I
had
similar problem before, and even if I only used
evaluate()
method
and load the template using my own java program, it happened
when the
for loop is lot of iterations.
The bottom line for my problem is I need to turn on template
caching,
so, velocity will not try to cache the objects it parsed
during each
time of the iteration.
I did:
ri.setProperty(,
"true");
where ri is a runtime instance.
Will this fix your issue?
Cheers,
Jian
5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
wrote:
I am getting an MemoryException:
Invocation of
method 'get' in class
threw exception class MemoryError : null
when I am creating a large velocity file. The vm file
itself
is
fairly small, but this piece of velocity is causing the
exception:
foreach($row in $List)
<tr>
<td height="20" class="tbltxt">$!row.item1</td>
<td class="tbltxt">$!row.item2</td>
<td class="tbltxt">$!row.item3</td>
<td class="tbltxt">$!row.item4</td>
<td class="tbltxt">$!number.format('currency',
$!row.item5)</td>
</tr>
end
List is a list of relatively small objects (5-6 Strings, a
couple of
BigDecimals), but the List itself contains approximately
66,000
entries. Unfortunately, unless I set the tomcat JVM to use
a
heap
size of at least 256M, this velocity template throws an
MemoryException when it is being processed.
Any suggestions?
Thanks.
L
--No.2 | | 7050 bytes |
| now I'm confused (suprise, suprise!). I looked at the source for VelocityServlet, and mergeTemplate has:
vw = (VelocityWriter) writerPool.get();
if (vw == null)
{
vw = new VelocityWriter(new StreamWriter(output, encoding),4 * 1024, true);
}
else
{
vw.recycle(new StreamWriter(output, encoding));
}
template.merge(context, vw);
So the VelocityServlet DES use StreamWriter already so then why does it still utilize so much memory?
-L
Fri, May 06, 2005 at 02:02:07PM -0700, lawrence (AT) marchex (DOT) com wrote:
Alright, I changed the Writer to be:
StreamWriter w = new StreamWriter(new FStream("test.html"));
And this fixes the memory problem.
However, my application is running Velocity as a servlet, how should I extend the VelocityServlet to use an StreamWriter intead of a StringWriter? Should I post a different email thread about this?
-L
Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
Well, a file, use a FileWriter.
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:48 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
I printed out the available memory before and after I call
"mergeTemplate":
Free memory before merge: 217393352
Free memory after merge: 16271392
so I would assume the StringWriter is approximately 200MB after the
merge. What should I use/do instead?
-Lawrence
Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
Why do you use StringWriter? That is cached in the memory.
How big is the StringWriter after the merge?
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:22 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
Alright, I ran this little piece of code:
public static void main( String args[] ) throws Exception
{
/* first, we init the runtime engine. Defaults are fine. */
Velocity.init();
/* lets make a Context and put data into it */
VelocityContext context = new VelocityContext();
List results = ReportBean.runLargeReport();
context.put("List", results);
/* lets render a template */
StringWriter w = new StringWriter();
Velocity.mergeTemplate("/myfile.vm", context, w );
System.out.println(" template : " + w );
/* lets make our own string to render */
}
results is a List of 66000 objects and requires about 8megs of memory.
The "mergeTemplate" requires at least 230 MB of memory. Any
suggestions?
-L
Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
Hi,
I guess the writer Ilkka talked about is when using Velocity as a
standalone template engine.
Right now I don't have a good clue, could you run Velocity as a
standalone template engine and render the page to see if there is a
huge memory usage?
Cheers,
Jian
5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
Sorry, I'm not sure what you mean by what "kind of writer" I'm
using. Where would I find out? How would I change this (to maybe a
StringBuffer writer?)
Also, each column contains a String that is at most, 200
characters,
most are closer to 10-20. There are a total of 10 columns.
Thanks.
-Lawrence
Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
What kind of writer are you using and how long are the items? 66
000 HTML
table rows with 5 numeric columns consumes about 32 MB when
buffered into a
string writer. If the items are longer, it takes much more
memory.
-- Ilkka
--
lawrence (AT) marchex (DOT) com wrote:
>I tried putting static HTML in the #foreach loop, same problem.
Then I
>tried setting the large list as a "local" varible:
>
>#set ($localList = $List)
>#foreach($row in $localList)
>
>
>but that didnt help either. I don't see why velocity should
use
250+ MB
>of memory iterating over a list ~8MB in size
>-L
>
Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
>
>>Well, I don't have a good clue now.
>>
>>What about do the debugging step by step, i.e., use just
static
html
>>in your #foreach loop? Will that blow up the memory?
>>
>>Cheers,
>>
>>Jian
>>
>>
>>
>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
>>
Thanks for the suggestion. I tried putting that, along with
file.resource.loader.cache = true
in my velocity.properties file. Both did not work. Any
other
suggestions?
Thanks!
L
Thu, May 05, 2005 at 02:11:28PM -0700, jian chen wrote:
Hi,
This maybe because you haven't turned on template caching. I
had
similar problem before, and even if I only used
evaluate()
method
and load the template using my own java program, it happened
when the
for loop is lot of iterations.
The bottom line for my problem is I need to turn on template
caching,
so, velocity will not try to cache the objects it parsed
during each
time of the iteration.
I did:
ri.setProperty(,
"true");
where ri is a runtime instance.
Will this fix your issue?
Cheers,
Jian
5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
wrote:
I am getting an MemoryException:
Invocation of
method 'get' in class
threw exception class MemoryError : null
when I am creating a large velocity file. The vm file
itself
is
fairly small, but this piece of velocity is causing the
exception:
foreach($row in $List)
<tr>
<td height="20" class="tbltxt">$!row.item1</td>
<td class="tbltxt">$!row.item2</td>
<td class="tbltxt">$!row.item3</td>
<td class="tbltxt">$!row.item4</td>
<td class="tbltxt">$!number.format('currency',
$!row.item5)</td>
</tr>
end
List is a list of relatively small objects (5-6 Strings, a
couple of
BigDecimals), but the List itself contains approximately
66,000
entries. Unfortunately, unless I set the tomcat JVM to use
a
heap
size of at least 256M, this velocity template throws an
MemoryException when it is being processed.
Any suggestions?
Thanks.
L
--No.3 | | 14069 bytes |
| Hi Lawrence,
I got interested, and just ran some similar code with JProfiler looking at
VM memory usage. I created a list of 60,000 objects, each with 10 100char
columns. Then I used StringWriter to merge as you did.
The VM used 3MB at the start, 6.5 MB after creating the list, and 25 after
merging. (before I closed the writer). Nowhere close to 200. So perhaps
there's something else going on. Although if the problem is one with
StringWriter, a lot may depend on the specific JVM you're using. (I'm using
Sun 1.4.2_04-b05).
WILL
Message
From: <lawrence (AT) marchex (DOT) com>
To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
Sent: Friday, May 06, 2005 2:08 PM
Subject: Re: MemoryException Processing Velocity Template
now I'm confused (suprise, suprise!). I looked at the source for
VelocityServlet, and mergeTemplate has:
vw = (VelocityWriter) writerPool.get();
if (vw == null)
{
vw = new VelocityWriter(new StreamWriter(output, encoding),4 *
1024, true);
}
else
{
vw.recycle(new StreamWriter(output, encoding));
}
template.merge(context, vw);
So the VelocityServlet DES use StreamWriter already so then why
does it still utilize so much memory?
-L
Fri, May 06, 2005 at 02:02:07PM -0700, lawrence (AT) marchex (DOT) com wrote:
>Alright, I changed the Writer to be:
>>
>StreamWriter w = new StreamWriter(new
>FStream("test.html"));
>>
>And this fixes the memory problem.
>>
>However, my application is running Velocity as a servlet, how should I
>extend the VelocityServlet to use an StreamWriter intead of a
>StringWriter? Should I post a different email thread about this?
>>
>-L
>>
>Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
>Well, a file, use a FileWriter.
>>
>David
>>
>>
>Message
>From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
>Sent: Friday, May 06, 2005 1:48 PM
>To: Velocity Users List
>Subject: Re: MemoryException Processing Velocity Template
>>
>I printed out the available memory before and after I call
>"mergeTemplate":
>>
>Free memory before merge: 217393352
>Free memory after merge: 16271392
>>
>so I would assume the StringWriter is approximately 200MB after the
>merge. What should I use/do instead?
>>
>-Lawrence
>>
>Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
>Why do you use StringWriter? That is cached in the memory.
>How big is the StringWriter after the merge?
>>
>David
>>
>>
>Message
>From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
>Sent: Friday, May 06, 2005 1:22 PM
>To: Velocity Users List
>Subject: Re: MemoryException Processing Velocity Template
>>
>Alright, I ran this little piece of code:
>>
>public static void main( String args[] ) throws Exception
>{
>/* first, we init the runtime engine. Defaults are fine. */
>>
>Velocity.init();
>>
>/* lets make a Context and put data into it */
>>
>VelocityContext context = new VelocityContext();
>>
>List results = ReportBean.runLargeReport();
>>
>context.put("List", results);
>>
>/* lets render a template */
>>
>StringWriter w = new StringWriter();
>>
>Velocity.mergeTemplate("/myfile.vm", context, w );
>System.out.println(" template : " + w );
>>
>/* lets make our own string to render */
>}
>>
>results is a List of 66000 objects and requires about 8megs of
>memory.
>The "mergeTemplate" requires at least 230 MB of memory. Any
>suggestions?
>>
>-L
>>
>Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
>Hi,
>>
>I guess the writer Ilkka talked about is when using Velocity as a
>standalone template engine.
>>
>Right now I don't have a good clue, could you run Velocity as a
>standalone template engine and render the page to see if there is a
>huge memory usage?
>>
>Cheers,
>>
>Jian
>>
>5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
>Sorry, I'm not sure what you mean by what "kind of writer" I'm
>using. Where would I find out? How would I change this (to maybe
>a
>StringBuffer writer?)
>>
>Also, each column contains a String that is at most, 200
>characters,
>most are closer to 10-20. There are a total of 10 columns.
>>
>Thanks.
>>
>-Lawrence
>>
>Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
>What kind of writer are you using and how long are the items?
>66
>000 HTML
>table rows with 5 numeric columns consumes about 32 MB when
>buffered into a
>string writer. If the items are longer, it takes much more
>memory.
>>
>-- Ilkka
>>
>>
>lawrence (AT) marchex (DOT) com wrote:
>>I tried putting static HTML in the #foreach loop, same
>>problem.
>Then I
>>tried setting the large list as a "local" varible:
>>
>>#set ($localList = $List)
>>#foreach($row in $localList)
>>
>>
>>but that didnt help either. I don't see why velocity should
>use
>250+ MB
>>of memory iterating over a list ~8MB in size
>>-L
>>
>Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
>>
>>>Well, I don't have a good clue now.
>>>
>>>What about do the debugging step by step, i.e., use just
>static
>html
>>>in your #foreach loop? Will that blow up the memory?
>>>
>>>Cheers,
>>>
>>>Jian
>>>
>>>
>>>
>>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
>>>
>Thanks for the suggestion. I tried putting that, along
>with
>
>file.resource.loader.cache = true
>
>in my velocity.properties file. Both did not work. Any
>other
>suggestions?
>
>Thanks!
>
>L
>
>Thu, May 05, 2005 at 02:11:28PM -0700, jian chen wrote:
>
>Hi,
>
>This maybe because you haven't turned on template caching.
>I
>had
>similar problem before, and even if I only used
>evaluate()
>method
>and load the template using my own java program, it
>happened
>when the
>for loop is lot of iterations.
>
>The bottom line for my problem is I need to turn on
>template
>caching,
>so, velocity will not try to cache the objects it parsed
>during each
>time of the iteration.
>
>I did:
>ri.setProperty(,
>"true");
>
>where ri is a runtime instance.
>
>Will this fix your issue?
>
>Cheers,
>
>Jian
>
>
>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
>wrote:
>
>I am getting an MemoryException:
>
>
>Invocation of
>method 'get' in class
>
>threw exception class MemoryError : null
>
>when I am creating a large velocity file. The vm file
>itself
>is
>fairly small, but this piece of velocity is causing the
>exception:
>
>foreach($row in $List)
><tr>
><td height="20" class="tbltxt">$!row.item1</td>
><td class="tbltxt">$!row.item2</td>
><td class="tbltxt">$!row.item3</td>
><td class="tbltxt">$!row.item4</td>
><td class="tbltxt">$!number.format('currency',
>$!row.item5)</td>
></tr>
>end
>
>List is a list of relatively small objects (5-6 Strings, a
>couple of
>BigDecimals), but the List itself contains approximately
>66,000
>entries. Unfortunately, unless I set the tomcat JVM to
>use
>a
>heap
>size of at least 256M, this velocity template throws an
>MemoryException when it is being processed.
>
>Any suggestions?
>
>Thanks.
>
>L
>
>>
>>
>
>--
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>
>>
>>
>
>-
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>>>
>>
>>
>>>
>>>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>>
>>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>.
>>
>>
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.orgNo.4 | | 9016 bytes |
| This is why I'm so confused. I wouldn't expect it to use so much memory. What I did to test the memory usage was:
System.out.println("Free memory before merge: " + Runtime.getRuntime().freeMemory());
Velocity.mergeTemplate("/myfile.vm", "IS", context, w );
System.out.println("Free memory after merge: " + Runtime.getRuntime().freeMemory());
And then setting the JVM to -Xms256M and -Xmx256M. When I use a String writer, the free memory is reduced by ~200MB. When I use an StreamWriter, it is reduced ~13MB, a huge difference.
BTW, I am using 1.4.2_05
-L
Fri, May 06, 2005 at 02:22:01PM -0700, Will Glass-Husain wrote:
Hi Lawrence,
I got interested, and just ran some similar code with JProfiler looking at
VM memory usage. I created a list of 60,000 objects, each with 10 100char
columns. Then I used StringWriter to merge as you did.
The VM used 3MB at the start, 6.5 MB after creating the list, and 25 after
merging. (before I closed the writer). Nowhere close to 200. So perhaps
there's something else going on. Although if the problem is one with
StringWriter, a lot may depend on the specific JVM you're using. (I'm
using Sun 1.4.2_04-b05).
WILL
Message
From: <lawrence (AT) marchex (DOT) com>
To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
Sent: Friday, May 06, 2005 2:08 PM
Subject: Re: MemoryException Processing Velocity Template
now I'm confused (suprise, suprise!). I looked at the source for
>VelocityServlet, and mergeTemplate has:
>
>vw = (VelocityWriter) writerPool.get();
>
>if (vw == null)
>{
vw = new VelocityWriter(new StreamWriter(output, encoding),4 *
>1024, true);
>}
>else
>{
vw.recycle(new StreamWriter(output, encoding));
>}
>template.merge(context, vw);
>
>
>So the VelocityServlet DES use StreamWriter already so then why
>does it still utilize so much memory?
>
>-L
>
Fri, May 06, 2005 at 02:02:07PM -0700, lawrence (AT) marchex (DOT) com wrote:
>>Alright, I changed the Writer to be:
>>
>>StreamWriter w = new StreamWriter(new
>>FStream("test.html"));
>>
>>And this fixes the memory problem.
>>
>>However, my application is running Velocity as a servlet, how should I
>>extend the VelocityServlet to use an StreamWriter intead of a
>>StringWriter? Should I post a different email thread about this?
>>
L
>>
>Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
Well, a file, use a FileWriter.
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:48 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
I printed out the available memory before and after I call
"mergeTemplate":
Free memory before merge: 217393352
Free memory after merge: 16271392
so I would assume the StringWriter is approximately 200MB after the
merge. What should I use/do instead?
-Lawrence
Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
Why do you use StringWriter? That is cached in the memory.
How big is the StringWriter after the merge?
David
--
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:22 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
Alright, I ran this little piece of code:
public static void main( String args[] ) throws Exception
{
/* first, we init the runtime engine. Defaults are fine. */
Velocity.init();
/* lets make a Context and put data into it */
VelocityContext context = new VelocityContext();
List results = ReportBean.runLargeReport();
context.put("List", results);
/* lets render a template */
StringWriter w = new StringWriter();
Velocity.mergeTemplate("/myfile.vm", context, w );
System.out.println(" template : " + w );
/* lets make our own string to render */
}
results is a List of 66000 objects and requires about 8megs of
memory.
The "mergeTemplate" requires at least 230 MB of memory. Any
suggestions?
-L
Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
Hi,
I guess the writer Ilkka talked about is when using Velocity as a
standalone template engine.
Right now I don't have a good clue, could you run Velocity as a
standalone template engine and render the page to see if there is a
huge memory usage?
Cheers,
Jian
5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
Sorry, I'm not sure what you mean by what "kind of writer" I'm
using. Where would I find out? How would I change this (to maybe
a
StringBuffer writer?)
Also, each column contains a String that is at most, 200
characters,
most are closer to 10-20. There are a total of 10 columns.
Thanks.
-Lawrence
Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
What kind of writer are you using and how long are the items?
66
000 HTML
table rows with 5 numeric columns consumes about 32 MB when
buffered into a
string writer. If the items are longer, it takes much more
memory.
-- Ilkka
--
lawrence (AT) marchex (DOT) com wrote:
>I tried putting static HTML in the #foreach loop, same
>problem.
Then I
>tried setting the large list as a "local" varible:
>
>#set ($localList = $List)
>#foreach($row in $localList)
>
>
>but that didnt help either. I don't see why velocity should
use
250+ MB
>of memory iterating over a list ~8MB in size
>-L
>
Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
>
>>Well, I don't have a good clue now.
>>
>>What about do the debugging step by step, i.e., use just
static
html
>>in your #foreach loop? Will that blow up the memory?
>>
>>Cheers,
>>
>>Jian
>>
>>
>>
>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
>>
Thanks for the suggestion. I tried putting that, along
with
file.resource.loader.cache = true
in my velocity.properties file. Both did not work. Any
other
suggestions?
Thanks!
L
Thu, May 05, 2005 at 02:11:28PM -0700, jian chen wrote:
Hi,
This maybe because you haven't turned on template caching.
I
had
similar problem before, and even if I only used
evaluate()
method
and load the template using my own java program, it
happened
when the
for loop is lot of iterations.
The bottom line for my problem is I need to turn on
template
caching,
so, velocity will not try to cache the objects it parsed
during each
time of the iteration.
I did:
ri.setProperty(,
"true");
where ri is a runtime instance.
Will this fix your issue?
Cheers,
Jian
5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
wrote:
I am getting an MemoryException:
Invocation of
method 'get' in class
threw exception class MemoryError : null
when I am creating a large velocity file. The vm file
itself
is
fairly small, but this piece of velocity is causing the
exception:
foreach($row in $List)
<tr>
<td height="20" class="tbltxt">$!row.item1</td>
<td class="tbltxt">$!row.item2</td>
<td class="tbltxt">$!row.item3</td>
<td class="tbltxt">$!row.item4</td>
<td class="tbltxt">$!number.format('currency',
$!row.item5)</td>
</tr>
end
List is a list of relatively small objects (5-6 Strings, a
couple of
BigDecimals), but the List itself contains approximately
66,000
entries. Unfortunately, unless I set the tomcat JVM to
use
a
heap
size of at least 256M, this velocity template throws an
MemoryException when it is being processed.
Any suggestions?
Thanks.
L
--No.5 | | 16283 bytes |
| You might consider trying a profiler. Most of them have trial version
available. This could tell you exactly where the memory is going.
I just bought JProfiler, and it seems a bit better priced than the others.
Here's a user review:
You might also check out Hyades from the Eclipse project for an open source
option.
WILL
Message
From: <lawrence (AT) marchex (DOT) com>
To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
Sent: Friday, May 06, 2005 2:39 PM
Subject: Re: MemoryException Processing Velocity Template
This is why I'm so confused. I wouldn't expect it to use so much memory.
What I did to test the memory usage was:
System.out.println("Free memory before merge: " +
Runtime.getRuntime().freeMemory());
Velocity.mergeTemplate("/myfile.vm", "IS", context, w );
System.out.println("Free memory after merge: " +
Runtime.getRuntime().freeMemory());
And then setting the JVM to -Xms256M and -Xmx256M. When I use a String
writer, the free memory is reduced by ~200MB. When I use an
StreamWriter, it is reduced ~13MB, a huge difference.
BTW, I am using 1.4.2_05
-L
Fri, May 06, 2005 at 02:22:01PM -0700, Will Glass-Husain wrote:
>Hi Lawrence,
>>
>I got interested, and just ran some similar code with JProfiler looking
>at
>VM memory usage. I created a list of 60,000 objects, each with 10
>100char
>columns. Then I used StringWriter to merge as you did.
>>
>The VM used 3MB at the start, 6.5 MB after creating the list, and 25
>after
>merging. (before I closed the writer). Nowhere close to 200. So
>perhaps
>there's something else going on. Although if the problem is one with
>StringWriter, a lot may depend on the specific JVM you're using. (I'm
>using Sun 1.4.2_04-b05).
>>
>WILL
>>
>Message
>From: <lawrence (AT) marchex (DOT) com>
>To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
>Sent: Friday, May 06, 2005 2:08 PM
>Subject: Re: MemoryException Processing Velocity Template
>>
>>
>now I'm confused (suprise, suprise!). I looked at the source for
>>VelocityServlet, and mergeTemplate has:
>>
>>vw = (VelocityWriter) writerPool.get();
>>
>>if (vw == null)
>>{
>vw = new VelocityWriter(new StreamWriter(output, encoding),4 *
>>1024, true);
>>}
>>else
>>{
>vw.recycle(new StreamWriter(output, encoding));
>>}
>>template.merge(context, vw);
>>
>
>>
>>So the VelocityServlet DES use StreamWriter already so then
>>why
>>does it still utilize so much memory?
>>
>>-L
>>
>Fri, May 06, 2005 at 02:02:07PM -0700, lawrence (AT) marchex (DOT) com wrote:
>>>Alright, I changed the Writer to be:
>>>
>>>StreamWriter w = new StreamWriter(new
>>>FStream("test.html"));
>>>
>>>And this fixes the memory problem.
>>>
>>>However, my application is running Velocity as a servlet, how should I
>>>extend the VelocityServlet to use an StreamWriter intead of a
>>>StringWriter? Should I post a different email thread about this?
>>>
>L
>>>
>>Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
>Well, a file, use a FileWriter.
>
>David
>
>
>Message
>From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
>Sent: Friday, May 06, 2005 1:48 PM
>To: Velocity Users List
>Subject: Re: MemoryException Processing Velocity Template
>
>I printed out the available memory before and after I call
>"mergeTemplate":
>
>Free memory before merge: 217393352
>Free memory after merge: 16271392
>
>so I would assume the StringWriter is approximately 200MB after the
>merge. What should I use/do instead?
>
>-Lawrence
>
>Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
>Why do you use StringWriter? That is cached in the memory.
>How big is the StringWriter after the merge?
>>
>David
>>
>>
>Message
>From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
>Sent: Friday, May 06, 2005 1:22 PM
>To: Velocity Users List
>Subject: Re: MemoryException Processing Velocity Template
>>
>Alright, I ran this little piece of code:
>>
>public static void main( String args[] ) throws Exception
>{
>/* first, we init the runtime engine. Defaults are fine.
>*/
>>
>Velocity.init();
>>
>/* lets make a Context and put data into it */
>>
>VelocityContext context = new VelocityContext();
>>
>List results = ReportBean.runLargeReport();
>>
>context.put("List", results);
>>
>/* lets render a template */
>>
>StringWriter w = new StringWriter();
>>
>Velocity.mergeTemplate("/myfile.vm", context, w );
>System.out.println(" template : " + w );
>>
>/* lets make our own string to render */
>}
>>
>results is a List of 66000 objects and requires about 8megs of
>memory.
>The "mergeTemplate" requires at least 230 MB of memory. Any
>suggestions?
>>
>-L
>>
>Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
>Hi,
>>
>I guess the writer Ilkka talked about is when using Velocity as a
>standalone template engine.
>>
>Right now I don't have a good clue, could you run Velocity as a
>standalone template engine and render the page to see if there is
>a
>huge memory usage?
>>
>Cheers,
>>
>Jian
>>
>5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
>Sorry, I'm not sure what you mean by what "kind of writer" I'm
>using. Where would I find out? How would I change this (to
>maybe
>a
>StringBuffer writer?)
>>
>Also, each column contains a String that is at most, 200
>characters,
>most are closer to 10-20. There are a total of 10 columns.
>>
>Thanks.
>>
>-Lawrence
>>
>Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
>What kind of writer are you using and how long are the items?
>66
>000 HTML
>table rows with 5 numeric columns consumes about 32 MB when
>buffered into a
>string writer. If the items are longer, it takes much more
>memory.
>>
>-- Ilkka
>>
>>
>lawrence (AT) marchex (DOT) com wrote:
>>I tried putting static HTML in the #foreach loop, same
>>problem.
>Then I
>>tried setting the large list as a "local" varible:
>>
>>#set ($localList = $List)
>>#foreach($row in $localList)
>>
>>
>>but that didnt help either. I don't see why velocity
>>should
>use
>250+ MB
>>of memory iterating over a list ~8MB in size
>>-L
>>
>Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
>>
>>>Well, I don't have a good clue now.
>>>
>>>What about do the debugging step by step, i.e., use just
>static
>html
>>>in your #foreach loop? Will that blow up the memory?
>>>
>>>Cheers,
>>>
>>>Jian
>>>
>>>
>>>
>>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com
>>>wrote:
>>>
>Thanks for the suggestion. I tried putting that, along
>with
>
>file.resource.loader.cache = true
>
>in my velocity.properties file. Both did not work. Any
>other
>suggestions?
>
>Thanks!
>
>L
>
>Thu, May 05, 2005 at 02:11:28PM -0700, jian chen wrote:
>
>Hi,
>
>This maybe because you haven't turned on template
>caching.
>I
>had
>similar problem before, and even if I only used
>evaluate()
>method
>and load the template using my own java program, it
>happened
>when the
>for loop is lot of iterations.
>
>The bottom line for my problem is I need to turn on
>template
>caching,
>so, velocity will not try to cache the objects it parsed
>during each
>time of the iteration.
>
>I did:
>ri.setProperty(,
>"true");
>
>where ri is a runtime instance.
>
>Will this fix your issue?
>
>Cheers,
>
>Jian
>
>
>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
>wrote:
>
>I am getting an MemoryException:
>
>
>Invocation of
>method 'get' in class
>
>threw exception class MemoryError : null
>
>when I am creating a large velocity file. The vm file
>itself
>is
>fairly small, but this piece of velocity is causing the
>exception:
>
>foreach($row in $List)
><tr>
><td height="20" class="tbltxt">$!row.item1</td>
><td class="tbltxt">$!row.item2</td>
><td class="tbltxt">$!row.item3</td>
><td class="tbltxt">$!row.item4</td>
><td class="tbltxt">$!number.format('currency',
>$!row.item5)</td>
></tr>
>end
>
>List is a list of relatively small objects (5-6 Strings,
>a
>couple of
>BigDecimals), but the List itself contains approximately
>66,000
>entries. Unfortunately, unless I set the tomcat JVM to
>use
>a
>heap
>size of at least 256M, this velocity template throws an
>MemoryException when it is being processed.
>
>Any suggestions?
>
>Thanks.
>
>L
>
>>
>>
>
>--
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>
>>
>>
>
>-
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>>>
>>
>>
>>>
>>>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>>
>>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>.
>>
>>
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>>
>>>
>>>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.orgNo.6 | | 9931 bytes |
| I tried using JProfiler and used a StringWriter, and found that the item that was taking the most memory (150 MB) was char[]! now I'm completely stumped and clueless.
-L
Fri, May 06, 2005 at 02:51:15PM -0700, Will Glass-Husain wrote:
You might consider trying a profiler. Most of them have trial version
available. This could tell you exactly where the memory is going.
I just bought JProfiler, and it seems a bit better priced than the others.
Here's a user review:
You might also check out Hyades from the Eclipse project for an open source
option.
WILL
Message
From: <lawrence (AT) marchex (DOT) com>
To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
Sent: Friday, May 06, 2005 2:39 PM
Subject: Re: MemoryException Processing Velocity Template
>This is why I'm so confused. I wouldn't expect it to use so much memory.
>What I did to test the memory usage was:
>
>System.out.println("Free memory before merge: " +
>Runtime.getRuntime().freeMemory());
>Velocity.mergeTemplate("/myfile.vm", "IS", context, w );
>System.out.println("Free memory after merge: " +
>Runtime.getRuntime().freeMemory());
>
>And then setting the JVM to -Xms256M and -Xmx256M. When I use a String
>writer, the free memory is reduced by ~200MB. When I use an
>StreamWriter, it is reduced ~13MB, a huge difference.
>
>BTW, I am using 1.4.2_05
>
>-L
>
Fri, May 06, 2005 at 02:22:01PM -0700, Will Glass-Husain wrote:
>>Hi Lawrence,
>>
>>I got interested, and just ran some similar code with JProfiler looking
>>at
>>VM memory usage. I created a list of 60,000 objects, each with 10
>>100char
>>columns. Then I used StringWriter to merge as you did.
>>
>>The VM used 3MB at the start, 6.5 MB after creating the list, and 25
>>after
>>merging. (before I closed the writer). Nowhere close to 200. So
>>perhaps
>>there's something else going on. Although if the problem is one with
>>StringWriter, a lot may depend on the specific JVM you're using. (I'm
>>using Sun 1.4.2_04-b05).
>>
>>WILL
>>
>Message
>>From: <lawrence (AT) marchex (DOT) com>
>>To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
>>Sent: Friday, May 06, 2005 2:08 PM
>>Subject: Re: MemoryException Processing Velocity Template
>>
>>
now I'm confused (suprise, suprise!). I looked at the source for
VelocityServlet, and mergeTemplate has:
vw = (VelocityWriter) writerPool.get();
if (vw == null)
{
vw = new VelocityWriter(new StreamWriter(output, encoding),4 *
1024, true);
}
else
{
vw.recycle(new StreamWriter(output, encoding));
}
template.merge(context, vw);
So the VelocityServlet DES use StreamWriter already so then
why
does it still utilize so much memory?
L
Fri, May 06, 2005 at 02:02:07PM -0700, lawrence (AT) marchex (DOT) com wrote:
Alright, I changed the Writer to be:
StreamWriter w = new StreamWriter(new
FStream("test.html"));
And this fixes the memory problem.
However, my application is running Velocity as a servlet, how should I
extend the VelocityServlet to use an StreamWriter intead of a
StringWriter? Should I post a different email thread about this?
L
Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
Well, a file, use a FileWriter.
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:48 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
I printed out the available memory before and after I call
"mergeTemplate":
Free memory before merge: 217393352
Free memory after merge: 16271392
so I would assume the StringWriter is approximately 200MB after the
merge. What should I use/do instead?
-Lawrence
Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
Why do you use StringWriter? That is cached in the memory.
How big is the StringWriter after the merge?
David
--
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:22 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
Alright, I ran this little piece of code:
public static void main( String args[] ) throws Exception
{
/* first, we init the runtime engine. Defaults are fine.
*/
Velocity.init();
/* lets make a Context and put data into it */
VelocityContext context = new VelocityContext();
List results = ReportBean.runLargeReport();
context.put("List", results);
/* lets render a template */
StringWriter w = new StringWriter();
Velocity.mergeTemplate("/myfile.vm", context, w );
System.out.println(" template : " + w );
/* lets make our own string to render */
}
results is a List of 66000 objects and requires about 8megs of
memory.
The "mergeTemplate" requires at least 230 MB of memory. Any
suggestions?
-L
Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
Hi,
I guess the writer Ilkka talked about is when using Velocity as a
standalone template engine.
Right now I don't have a good clue, could you run Velocity as a
standalone template engine and render the page to see if there is
a
huge memory usage?
Cheers,
Jian
5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
Sorry, I'm not sure what you mean by what "kind of writer" I'm
using. Where would I find out? How would I change this (to
maybe
a
StringBuffer writer?)
Also, each column contains a String that is at most, 200
characters,
most are closer to 10-20. There are a total of 10 columns.
Thanks.
-Lawrence
Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
What kind of writer are you using and how long are the items?
66
000 HTML
table rows with 5 numeric columns consumes about 32 MB when
buffered into a
string writer. If the items are longer, it takes much more
memory.
-- Ilkka
--
lawrence (AT) marchex (DOT) com wrote:
>I tried putting static HTML in the #foreach loop, same
>problem.
Then I
>tried setting the large list as a "local" varible:
>
>#set ($localList = $List)
>#foreach($row in $localList)
>
>
>but that didnt help either. I don't see why velocity
>should
use
250+ MB
>of memory iterating over a list ~8MB in size
>-L
>
Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
>
>>Well, I don't have a good clue now.
>>
>>What about do the debugging step by step, i.e., use just
static
html
>>in your #foreach loop? Will that blow up the memory?
>>
>>Cheers,
>>
>>Jian
>>
>>
>>
>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com
>>wrote:
>>
Thanks for the suggestion. I tried putting that, along
with
file.resource.loader.cache = true
in my velocity.properties file. Both did not work. Any
other
suggestions?
Thanks!
L
Thu, May 05, 2005 at 02:11:28PM -0700, jian chen wrote:
Hi,
This maybe because you haven't turned on template
caching.
I
had
similar problem before, and even if I only used
evaluate()
method
and load the template using my own java program, it
happened
when the
for loop is lot of iterations.
The bottom line for my problem is I need to turn on
template
caching,
so, velocity will not try to cache the objects it parsed
during each
time of the iteration.
I did:
ri.setProperty(,
"true");
where ri is a runtime instance.
Will this fix your issue?
Cheers,
Jian
5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
wrote:
I am getting an MemoryException:
Invocation of
method 'get' in class
threw exception class MemoryError : null
when I am creating a large velocity file. The vm file
itself
is
fairly small, but this piece of velocity is causing the
exception:
foreach($row in $List)
<tr>
<td height="20" class="tbltxt">$!row.item1</td>
<td class="tbltxt">$!row.item2</td>
<td class="tbltxt">$!row.item3</td>
<td class="tbltxt">$!row.item4</td>
<td class="tbltxt">$!number.format('currency',
$!row.item5)</td>
</tr>
end
List is a list of relatively small objects (5-6 Strings,
a
couple of
BigDecimals), but the List itself contains approximately
66,000
entries. Unfortunately, unless I set the tomcat JVM to
use
a
heap
size of at least 256M, this velocity template throws an
MemoryException when it is being processed.
Any suggestions?
Thanks.
L
--No.7 | | 17989 bytes |
| Yes, but which ones?
The char[] in the StringWriter, the char[] in the Template object, or the
char[] in the strings in your original objects? That would be useful info.
WILL
Message
From: <lawrence (AT) marchex (DOT) com>
To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
Sent: Friday, May 06, 2005 3:45 PM
Subject: Re: MemoryException Processing Velocity Template
I tried using JProfiler and used a StringWriter, and found that the
item that was taking the most memory (150 MB) was char[]! now I'm
completely stumped and clueless.
-L
Fri, May 06, 2005 at 02:51:15PM -0700, Will Glass-Husain wrote:
>You might consider trying a profiler. Most of them have trial version
>available. This could tell you exactly where the memory is going.
>>
>I just bought JProfiler, and it seems a bit better priced than the
>others.
>Here's a user review:
>
>>
>You might also check out Hyades from the Eclipse project for an open
>source
>option.
>>
>WILL
>>
>>
>Message
>From: <lawrence (AT) marchex (DOT) com>
>To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
>Sent: Friday, May 06, 2005 2:39 PM
>Subject: Re: MemoryException Processing Velocity Template
>>
>>
>>This is why I'm so confused. I wouldn't expect it to use so much
>>memory.
>>What I did to test the memory usage was:
>>
>>System.out.println("Free memory before merge: " +
>>Runtime.getRuntime().freeMemory());
>>Velocity.mergeTemplate("/myfile.vm", "IS", context, w );
>>System.out.println("Free memory after merge: " +
>>Runtime.getRuntime().freeMemory());
>>
>>And then setting the JVM to -Xms256M and -Xmx256M. When I use a String
>>writer, the free memory is reduced by ~200MB. When I use an
>>StreamWriter, it is reduced ~13MB, a huge difference.
>>
>>BTW, I am using 1.4.2_05
>>
>>-L
>>
>Fri, May 06, 2005 at 02:22:01PM -0700, Will Glass-Husain wrote:
>>>Hi Lawrence,
>>>
>>>I got interested, and just ran some similar code with JProfiler looking
>>>at
>>>VM memory usage. I created a list of 60,000 objects, each with 10
>>>100char
>>>columns. Then I used StringWriter to merge as you did.
>>>
>>>The VM used 3MB at the start, 6.5 MB after creating the list, and 25
>>>after
>>>merging. (before I closed the writer). Nowhere close to 200. So
>>>perhaps
>>>there's something else going on. Although if the problem is one with
>>>StringWriter, a lot may depend on the specific JVM you're using. (I'm
>>>using Sun 1.4.2_04-b05).
>>>
>>>WILL
>>>
>>Message
>>>From: <lawrence (AT) marchex (DOT) com>
>>>To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
>>>Sent: Friday, May 06, 2005 2:08 PM
>>>Subject: Re: MemoryException Processing Velocity Template
>>>
>>>
>now I'm confused (suprise, suprise!). I looked at the source for
>VelocityServlet, and mergeTemplate has:
>
>vw = (VelocityWriter) writerPool.get();
>
>if (vw == null)
>{
>vw = new VelocityWriter(new StreamWriter(output, encoding),4
>*
>1024, true);
>}
>else
>{
>vw.recycle(new StreamWriter(output, encoding));
>}
>template.merge(context, vw);
>
>
>
>So the VelocityServlet DES use StreamWriter already so then
>why
>does it still utilize so much memory?
>
>L
>
>Fri, May 06, 2005 at 02:02:07PM -0700, lawrence (AT) marchex (DOT) com wrote:
>Alright, I changed the Writer to be:
>
>StreamWriter w = new StreamWriter(new
>FStream("test.html"));
>
>And this fixes the memory problem.
>
>However, my application is running Velocity as a servlet, how should
>I
>extend the VelocityServlet to use an StreamWriter intead of a
>StringWriter? Should I post a different email thread about this?
>
>L
>
>Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
>Well, a file, use a FileWriter.
>
>David
>
>
>Message
>From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
>Sent: Friday, May 06, 2005 1:48 PM
>To: Velocity Users List
>Subject: Re: MemoryException Processing Velocity Template
>
>I printed out the available memory before and after I call
>"mergeTemplate":
>
>Free memory before merge: 217393352
>Free memory after merge: 16271392
>
>so I would assume the StringWriter is approximately 200MB after the
>merge. What should I use/do instead?
>
>-Lawrence
>
>Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
>Why do you use StringWriter? That is cached in the memory.
>How big is the StringWriter after the merge?
>>
>David
>>
>>
>Message
>From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
>Sent: Friday, May 06, 2005 1:22 PM
>To: Velocity Users List
>Subject: Re: MemoryException Processing Velocity Template
>>
>Alright, I ran this little piece of code:
>>
>public static void main( String args[] ) throws Exception
>{
>/* first, we init the runtime engine. Defaults are fine.
>*/
>>
>Velocity.init();
>>
>/* lets make a Context and put data into it */
>>
>VelocityContext context = new VelocityContext();
>>
>List results = ReportBean.runLargeReport();
>>
>context.put("List", results);
>>
>/* lets render a template */
>>
>StringWriter w = new StringWriter();
>>
>Velocity.mergeTemplate("/myfile.vm", context, w );
>System.out.println(" template : " + w );
>>
>/* lets make our own string to render */
>}
>>
>results is a List of 66000 objects and requires about 8megs of
>memory.
>The "mergeTemplate" requires at least 230 MB of memory. Any
>suggestions?
>>
>-L
>>
>Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
>Hi,
>>
>I guess the writer Ilkka talked about is when using Velocity as
>a
>standalone template engine.
>>
>Right now I don't have a good clue, could you run Velocity as a
>standalone template engine and render the page to see if there
>is
>a
>huge memory usage?
>>
>Cheers,
>>
>Jian
>>
>5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
>Sorry, I'm not sure what you mean by what "kind of writer"
>I'm
>using. Where would I find out? How would I change this (to
>maybe
>a
>StringBuffer writer?)
>>
>Also, each column contains a String that is at most, 200
>characters,
>most are closer to 10-20. There are a total of 10 columns.
>>
>Thanks.
>>
>-Lawrence
>>
>Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
>What kind of writer are you using and how long are the
>items?
>66
>000 HTML
>table rows with 5 numeric columns consumes about 32 MB when
>buffered into a
>string writer. If the items are longer, it takes much more
>memory.
>>
>-- Ilkka
>>
>>
>lawrence (AT) marchex (DOT) com wrote:
>>I tried putting static HTML in the #foreach loop, same
>>problem.
>Then I
>>tried setting the large list as a "local" varible:
>>
>>#set ($localList = $List)
>>#foreach($row in $localList)
>>
>>
>>but that didnt help either. I don't see why velocity
>>should
>use
>250+ MB
>>of memory iterating over a list ~8MB in size
>>-L
>>
>Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
>>
>>>Well, I don't have a good clue now.
>>>
>>>What about do the debugging step by step, i.e., use just
>static
>html
>>>in your #foreach loop? Will that blow up the memory?
>>>
>>>Cheers,
>>>
>>>Jian
>>>
>>>
>>>
>>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
>>>wrote:
>>>
>Thanks for the suggestion. I tried putting that, along
>with
>
>file.resource.loader.cache = true
>
>in my velocity.properties file. Both did not work.
>Any
>other
>suggestions?
>
>Thanks!
>
>L
>
>Thu, May 05, 2005 at 02:11:28PM -0700, jian chen
>wrote:
>
>Hi,
>
>This maybe because you haven't turned on template
>caching.
>I
>had
>similar problem before, and even if I only used
>evaluate()
>method
>and load the template using my own java program, it
>happened
>when the
>for loop is lot of iterations.
>
>The bottom line for my problem is I need to turn on
>template
>caching,
>so, velocity will not try to cache the objects it
>parsed
>during each
>time of the iteration.
>
>I did:
>>
>ri.setProperty(,
>"true");
>
>where ri is a runtime instance.
>
>Will this fix your issue?
>
>Cheers,
>
>Jian
>
>
>5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
>wrote:
>
>I am getting an MemoryException:
>
>
>Invocation of
>method 'get' in class
>
>threw exception class MemoryError :
>null
>
>when I am creating a large velocity file. The vm file
>itself
>is
>fairly small, but this piece of velocity is causing
>the
>exception:
>
>foreach($row in $List)
><tr>
><td height="20" class="tbltxt">$!row.item1</td>
><td class="tbltxt">$!row.item2</td>
><td class="tbltxt">$!row.item3</td>
><td class="tbltxt">$!row.item4</td>
><td class="tbltxt">$!number.format('currency',
>$!row.item5)</td>
></tr>
>end
>
>List is a list of relatively small objects (5-6
>Strings,
>a
>couple of
>BigDecimals), but the List itself contains
>approximately
>66,000
>entries. Unfortunately, unless I set the tomcat JVM
>to
>use
>a
>heap
>size of at least 256M, this velocity template throws
>an
>MemoryException when it is being processed.
>
>Any suggestions?
>
>Thanks.
>
>L
>
>>
>>
>
>
>--
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>
>>
>>
>
>
>-
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>>
>>
>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>>>
>>
>>
>
>
>>>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>>
>
>>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>.
>>
>>
>>
>>
>>
>>>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>>>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>>>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>
>To unsubscribe, e-mail:
>velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail:
>velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>
>>>
>>>
>>>
>>>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.orgNo.8 | | 10327 bytes |
| Maybe you are not referring to this memory eating template directly from
VelocityServlet, but through some tool, #include, etc., which uses a nested
StringWriter and not the servlet's StreamWriter directly. According to
your first post, the original oom exception was thrown by the Velocity Struts
Tiles tool
-- Ilkka
lawrence (AT) marchex (DOT) com wrote:
now I'm confused (suprise, suprise!). I looked at the source for VelocityServlet, and mergeTemplate has:
vw = (VelocityWriter) writerPool.get();
if (vw == null)
{
vw = new VelocityWriter(new StreamWriter(output, encoding),4 * 1024, true);
}
else
{
vw.recycle(new StreamWriter(output, encoding));
}
template.merge(context, vw);
So the VelocityServlet DES use StreamWriter already so then why does it still utilize so much memory?
-L
Fri, May 06, 2005 at 02:02:07PM -0700, lawrence (AT) marchex (DOT) com wrote:
>>Alright, I changed the Writer to be:
>>
>>StreamWriter w = new StreamWriter(new FStream("test.html"));
>>
>>And this fixes the memory problem.
>>
>>However, my application is running Velocity as a servlet, how should I extend the VelocityServlet to use an StreamWriter intead of a StringWriter? Should I post a different email thread about this?
>>
L
>>
>Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
>>
Well, a file, use a FileWriter.
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:48 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
I printed out the available memory before and after I call
"mergeTemplate":
Free memory before merge: 217393352
Free memory after merge: 16271392
so I would assume the StringWriter is approximately 200MB after the
merge. What should I use/do instead?
Lawrence
Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
Why do you use StringWriter? That is cached in the memory.
How big is the StringWriter after the merge?
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:22 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
Alright, I ran this little piece of code:
public static void main( String args[] ) throws Exception
{
/* first, we init the runtime engine. Defaults are fine. */
Velocity.init();
/* lets make a Context and put data into it */
VelocityContext context = new VelocityContext();
List results = ReportBean.runLargeReport();
context.put("List", results);
/* lets render a template */
StringWriter w = new StringWriter();
Velocity.mergeTemplate("/myfile.vm", context, w );
System.out.println(" template : " + w );
/* lets make our own string to render */
}
results is a List of 66000 objects and requires about 8megs of memory.
The "mergeTemplate" requires at least 230 MB of memory. Any
suggestions?
L
Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
Hi,
I guess the writer Ilkka talked about is when using Velocity as a
standalone template engine.
Right now I don't have a good clue, could you run Velocity as a
standalone template engine and render the page to see if there is a
huge memory usage?
Cheers,
Jian
5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
Sorry, I'm not sure what you mean by what "kind of writer" I'm
using. Where would I find out? How would I change this (to maybe a
StringBuffer writer?)
Also, each column contains a String that is at most, 200
characters,
most are closer to 10-20. There are a total of 10 columns.
Thanks.
Lawrence
Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
What kind of writer are you using and how long are the items? 66
000 HTML
table rows with 5 numeric columns consumes about 32 MB when
buffered into a
string writer. If the items are longer, it takes much more
memory.
Ilkka
lawrence (AT) marchex (DOT) com wrote:
I tried putting static HTML in the #foreach loop, same problem.
Then I
tried setting the large list as a "local" varible:
set ($localList = $List)
foreach($row in $localList)
but that didnt help either. I don't see why velocity should
use
250+ MB
of memory iterating over a list ~8MB in size
L
Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
Well, I don't have a good clue now.
What about do the debugging step by step, i.e., use just
static
html
in your #foreach loop? Will that blow up the memory?
Cheers,
Jian
5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
Thanks for the suggestion. I tried putting that, along with
file.resource.loader.cache = true
in my velocity.properties file. Both did not work. Any
other
suggestions?
Thanks!
L
Thu, May 05, 2005 at 02:11:28PM -0700, jian chen wrote:
Hi,
This maybe because you haven't turned on template caching. I
had
similar problem before, and even if I only used
evaluate()
method
and load the template using my own java program, it happened
when the
for loop is lot of iterations.
The bottom line for my problem is I need to turn on template
caching,
so, velocity will not try to cache the objects it parsed
during each
time of the iteration.
I did:
ri.setProperty(,
"true");
where ri is a runtime instance.
Will this fix your issue?
Cheers,
Jian
5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
wrote:
I am getting an MemoryException:
Invocation of
method 'get' in class
threw exception class MemoryError : null
when I am creating a large velocity file. The vm file
itself
is
fairly small, but this piece of velocity is causing the
exception:
foreach($row in $List)
<tr>
<td height="20" class="tbltxt">$!row.item1</td>
<td class="tbltxt">$!row.item2</td>
<td class="tbltxt">$!row.item3</td>
<td class="tbltxt">$!row.item4</td>
<td class="tbltxt">$!number.format('currency',
$!row.item5)</td>
</tr>
end
List is a list of relatively small objects (5-6 Strings, a
couple of
BigDecimals), but the List itself contains approximately
66,000
entries. Unfortunately, unless I set the tomcat JVM to use
a
heap
size of at least 256M, this velocity template throws an
MemoryException when it is being processed.
Any suggestions?
Thanks.
L
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
.
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.orgNo.9 | | 12889 bytes |
| The stack trace makes it seem like the problem still originates from the VelocityViewServlet, and not the TilesTool:
Invocation of method 'get' in class threw exception class MemoryError : null
at (ASTIdentifier.java:193)
at (ASTReference.java:175)
at (ASTReference.java:220)
at (SimpleNode.java:230)
at (Template.java:256)
at (TufteViewServlet.java:106)
at (Unknown Source)
at (Unknown Source)
at (HttpServlet.java:740)
at (HttpServlet.java:853)
at (ApplicationDispatcher.java:684)
at (ApplicationDispatcher.java:432)
at (ApplicationDispatcher.java:356)
at (RequestProcessor.java:1097)
at (TilesRequestProcessor.java:302)
at (TilesRequestProcessor.java:278)
at (TilesRequestProcessor.java:341)
at (RequestProcessor.java:272)
at (ActionServlet.java:1187)
at (ActionServlet.java:433)
at (HttpServlet.java:740)
at (HttpServlet.java:853)
at ()
at ()
at (StandardWrapperValve.java:256)
at $(StandardPipeline.java:643)
at (StandardPipeline.java:480)
at (ContainerBase.java:995)
at (StandardContextValve.java:191)
at $(StandardPipeline.java:643)
at (StandardPipeline.java:480)
at (ContainerBase.java:995)
at (StandardContext.java:2415)
at (StandardHostValve.java:180)
at $(StandardPipeline.java:643)
at (ErrorDispatcherValve.java:171)
at $(StandardPipeline.java:641)
at (ErrorReportValve.java:172)
at $(StandardPipeline.java:641)
at (StandardPipeline.java:480)
at (ContainerBase.java:995)
at (StandardEngineValve.java:174)
at $(StandardPipeline.java:643)
at (StandardPipeline.java:480)
at (ContainerBase.java:995)
at (CoyoteAdapter.java:223)
at (JkCoyoteHandler.java:261)
at (HandlerRequest.java:360)
at (ChannelSocket.java:604)
at (ChannelSocket.java:562)
at (ChannelSocket.java:679)
at $ControlRunnable.run(ThreadPool.java:619)
at java.lang.Thread.run(Thread.java:534)
So I suppose the question is, how effecient is SStream? When I use an FStream, very little memory is required.
-L
Sat, May 07, 2005 at 11:43:03AM +0300, Ilkka Priha wrote:
Maybe you are not referring to this memory eating template directly from
VelocityServlet, but through some tool, #include, etc., which uses a nested
StringWriter and not the servlet's StreamWriter directly. According
to your first post, the original oom exception was thrown by the Velocity
Struts Tiles tool
-- Ilkka
lawrence (AT) marchex (DOT) com wrote:
now I'm confused (suprise, suprise!). I looked at the source for
>VelocityServlet, and mergeTemplate has:
>
>vw = (VelocityWriter) writerPool.get();
>
>if (vw == null)
>{
vw = new VelocityWriter(new StreamWriter(output, encoding),4 *
1024, true);
>}
>else
>{
vw.recycle(new StreamWriter(output, encoding));
>}
>template.merge(context, vw);
>
>
>So the VelocityServlet DES use StreamWriter already so then why
>does it still utilize so much memory?
>
>-L
>
Fri, May 06, 2005 at 02:02:07PM -0700, lawrence (AT) marchex (DOT) com wrote:
>
>>Alright, I changed the Writer to be:
>>
>>StreamWriter w = new StreamWriter(new
>>FStream("test.html"));
>>
>>And this fixes the memory problem.
>>
>>However, my application is running Velocity as a servlet, how should I
>>extend the VelocityServlet to use an StreamWriter intead of a
>>StringWriter? Should I post a different email thread about this?
>>
L
>>
>Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
>>
Well, a file, use a FileWriter.
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:48 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
I printed out the available memory before and after I call
"mergeTemplate":
Free memory before merge: 217393352
Free memory after merge: 16271392
so I would assume the StringWriter is approximately 200MB after the
merge. What should I use/do instead?
Lawrence
Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
Why do you use StringWriter? That is cached in the memory.
How big is the StringWriter after the merge?
David
Message
From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
Sent: Friday, May 06, 2005 1:22 PM
To: Velocity Users List
Subject: Re: MemoryException Processing Velocity Template
Alright, I ran this little piece of code:
public static void main( String args[] ) throws Exception
{
/* first, we init the runtime engine. Defaults are fine. */
Velocity.init();
/* lets make a Context and put data into it */
VelocityContext context = new VelocityContext();
List results = ReportBean.runLargeReport();
context.put("List", results);
/* lets render a template */
StringWriter w = new StringWriter();
Velocity.mergeTemplate("/myfile.vm", context, w );
System.out.println(" template : " + w );
/* lets make our own string to render */
}
results is a List of 66000 objects and requires about 8megs of memory.
The "mergeTemplate" requires at least 230 MB of memory. Any
suggestions?
L
Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
Hi,
I guess the writer Ilkka talked about is when using Velocity as a
standalone template engine.
Right now I don't have a good clue, could you run Velocity as a
standalone template engine and render the page to see if there is a
huge memory usage?
Cheers,
Jian
5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
Sorry, I'm not sure what you mean by what "kind of writer" I'm
using. Where would I find out? How would I change this (to maybe a
StringBuffer writer?)
Also, each column contains a String that is at most, 200
characters,
most are closer to 10-20. There are a total of 10 columns.
Thanks.
Lawrence
Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
What kind of writer are you using and how long are the items? 66
000 HTML
table rows with 5 numeric columns consumes about 32 MB when
buffered into a
string writer. If the items are longer, it takes much more
memory.
Ilkka
lawrence (AT) marchex (DOT) com wrote:
I tried putting static HTML in the #foreach loop, same problem.
Then I
tried setting the large list as a "local" varible:
set ($localList = $List)
foreach($row in $localList)
but that didnt help either. I don't see why velocity should
use
250+ MB
of memory iterating over a list ~8MB in size
L
Thu, May 05, 2005 at 02:56:28PM -0700, jian chen wrote:
Well, I don't have a good clue now.
What about do the debugging step by step, i.e., use just
static
html
in your #foreach loop? Will that blow up the memory?
Cheers,
Jian
5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
Thanks for the suggestion. I tried putting that, along with
file.resource.loader.cache = true
in my velocity.properties file. Both did not work. Any
other
suggestions?
Thanks!
L
Thu, May 05, 2005 at 02:11:28PM -0700, jian chen wrote:
Hi,
This maybe because you haven't turned on template caching. I
had
similar problem before, and even if I only used
evaluate()
method
and load the template using my own java program, it happened
when the
for loop is lot of iterations.
The bottom line for my problem is I need to turn on template
caching,
so, velocity will not try to cache the objects it parsed
during each
time of the iteration.
I did:
ri.setProperty(,
"true");
where ri is a runtime instance.
Will this fix your issue?
Cheers,
Jian
5/5/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) com>
wrote:
I am getting an MemoryException:
Invocation of
method 'get' in class
threw exception class MemoryError : null
when I am creating a large velocity file. The vm file
itself
is
fairly small, but this piece of velocity is causing the
exception:
foreach($row in $List)
<tr>
<td height="20" class="tbltxt">$!row.item1</td>
<td class="tbltxt">$!row.item2</td>
<td class="tbltxt">$!row.item3</td>
<td class="tbltxt">$!row.item4</td>
<td class="tbltxt">$!number.format('currency',
$!row.item5)</td>
</tr>
end
List is a list of relatively small objects (5-6 Strings, a
couple of
BigDecimals), but the List itself contains approximately
66,000
entries. Unfortunately, unless I set the tomcat JVM to use
a
heap
size of at least 256M, this velocity template throws an
MemoryException when it is being processed.
Any suggestions?
Thanks.
L
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail:
velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail:
velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>>
>>
>>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>
>
>
>To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
>For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
>
>.
>
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.org
To unsubscribe, e-mail: velocity-user-unsubscribe (AT) jakarta (DOT) apache.org
For additional commands, e-mail: velocity-user-help (AT) jakarta (DOT) apache.orgNo.10 | | 15807 bytes |
| Not necessarily. Some other object might be hogging memory, but the system
finally runs out while executing the Velocity page.
WILL
Message
From: <lawrence (AT) marchex (DOT) com>
To: "Velocity Users List" <velocity-user (AT) jakarta (DOT) apache.org>
Sent: Monday, May 09, 2005 9:36 AM
Subject: Re: MemoryException Processing Velocity Template
The stack trace makes it seem like the problem still originates from the
VelocityViewServlet, and not the TilesTool:
Invocation of
method 'get' in class threw
exception class MemoryError : null
at
(ASTIdentifier.java:193)
at
(ASTReference.java:175)
at
(ASTReference.java:220)
at
(SimpleNode.java:230)
at (Template.java:256)
at
(TufteViewServlet.java:106)
at
(Unknown
Source)
at
(Unknown
Source)
at (HttpServlet.java:740)
at (HttpServlet.java:853)
at
(ApplicationDispatcher.java:684)
at
(ApplicationDispatcher.java:432)
at
(ApplicationDispatcher.java:356)
at
(RequestProcessor.java:1097)
at
(TilesRequestProcessor.java:302)
at
(TilesRequestProcessor.java:278)
at
(TilesRequestProcessor.java:341)
at
(RequestProcessor.java:272)
at (ActionServlet.java:1187)
at (ActionServlet.java:433)
at (HttpServlet.java:740)
at (HttpServlet.java:853)
at
()
at
()
at
(StandardWrapperValve.java:256)
at
$(StandardPipeline.java:643)
at
(StandardPipeline.java:480)
at (ContainerBase.java:995)
at
(StandardContextValve.java:191)
at
$(StandardPipeline.java:643)
at
(StandardPipeline.java:480)
at (ContainerBase.java:995)
at
(StandardContext.java:2415)
at
(StandardHostValve.java:180)
at
$(StandardPipeline.java:643)
at
(ErrorDispatcherValve.java:171)
at
$(StandardPipeline.java:641)
at
(ErrorReportValve.java:172)
at
$(StandardPipeline.java:641)
at
(StandardPipeline.java:480)
at (ContainerBase.java:995)
at
(StandardEngineValve.java:174)
at
$(StandardPipeline.java:643)
at
(StandardPipeline.java:480)
at (ContainerBase.java:995)
at (CoyoteAdapter.java:223)
at (JkCoyoteHandler.java:261)
at (HandlerRequest.java:360)
at (ChannelSocket.java:604)
at
(ChannelSocket.java:562)
at (ChannelSocket.java:679)
at
$ControlRunnable.run(ThreadPool.java:619)
at java.lang.Thread.run(Thread.java:534)
So I suppose the question is, how effecient is SStream? When
I use an FStream, very little memory is required.
-L
Sat, May 07, 2005 at 11:43:03AM +0300, Ilkka Priha wrote:
>Maybe you are not referring to this memory eating template directly from
>VelocityServlet, but through some tool, #include, etc., which uses a
>nested
>StringWriter and not the servlet's StreamWriter directly. According
>to your first post, the original oom exception was thrown by the Velocity
>Struts Tiles tool
>>
>-- Ilkka
>>
>>
>lawrence (AT) marchex (DOT) com wrote:
>now I'm confused (suprise, suprise!). I looked at the source for
>>VelocityServlet, and mergeTemplate has:
>>
>>vw = (VelocityWriter) writerPool.get();
>>
>>if (vw == null)
>>{
>vw = new VelocityWriter(new StreamWriter(output, encoding),4 *
>1024, true);
>>}
>>else
>>{
>vw.recycle(new StreamWriter(output, encoding));
>>}
>>template.merge(context, vw);
>>
>
>>
>>So the VelocityServlet DES use StreamWriter already so then
>>why
>>does it still utilize so much memory?
>>
>>-L
>>
>Fri, May 06, 2005 at 02:02:07PM -0700, lawrence (AT) marchex (DOT) com wrote:
>>
>>>Alright, I changed the Writer to be:
>>>
>>>StreamWriter w = new StreamWriter(new
>>>FStream("test.html"));
>>>
>>>And this fixes the memory problem.
>>>
>>>However, my application is running Velocity as a servlet, how should I
>>>extend the VelocityServlet to use an StreamWriter intead of a
>>>StringWriter? Should I post a different email thread about this?
>>>
>L
>>>
>>Fri, May 06, 2005 at 01:59:19PM -0700, David You wrote:
>>>
>Well, a file, use a FileWriter.
>
>David
>
>
>Message
>From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
>Sent: Friday, May 06, 2005 1:48 PM
>To: Velocity Users List
>Subject: Re: MemoryException Processing Velocity Template
>
>I printed out the available memory before and after I call
>"mergeTemplate":
>
>Free memory before merge: 217393352
>Free memory after merge: 16271392
>
>so I would assume the StringWriter is approximately 200MB after the
>merge. What should I use/do instead?
>
>Lawrence
>
>Fri, May 06, 2005 at 01:43:50PM -0700, David You wrote:
>
>Why do you use StringWriter? That is cached in the memory.
>How big is the StringWriter after the merge?
>
>David
>
>
>Message
>From: lawrence (AT) marchex (DOT) com [mailto:lawrence (AT) marchex (DOT) com]
>Sent: Friday, May 06, 2005 1:22 PM
>To: Velocity Users List
>Subject: Re: MemoryException Processing Velocity Template
>
>Alright, I ran this little piece of code:
>
>public static void main( String args[] ) throws Exception
>{
>/* first, we init the runtime engine. Defaults are fine. */
>
>Velocity.init();
>
>/* lets make a Context and put data into it */
>
>VelocityContext context = new VelocityContext();
>
>List results = ReportBean.runLargeReport();
>
>context.put("List", results);
>
>/* lets render a template */
>
>StringWriter w = new StringWriter();
>
>Velocity.mergeTemplate("/myfile.vm", context, w );
>System.out.println(" template : " + w );
>
>/* lets make our own string to render */
>}
>
>results is a List of 66000 objects and requires about 8megs of
>memory.
>The "mergeTemplate" requires at least 230 MB of memory. Any
>suggestions?
>
>L
>
>Fri, May 06, 2005 at 11:53:30AM -0700, jian chen wrote:
>
>Hi,
>
>I guess the writer Ilkka talked about is when using Velocity as a
>standalone template engine.
>
>Right now I don't have a good clue, could you run Velocity as a
>standalone template engine and render the page to see if there is a
>huge memory usage?
>
>Cheers,
>
>Jian
>
>5/6/05, lawrence (AT) marchex (DOT) com <lawrence (AT) marchex (DOT) comwrote:
>
>Sorry, I'm not sure what you mean by what "kind of writer" I'm
>
>using. Where would I find out? How would I change this (to maybe
>a
>StringBuffer writer?)
>
>Also, each column contains a String that is at most, 200
>
>characters,
>
>most are closer to 10-20. There are a total of 10 columns.
>
>Thanks.
>
>Lawrence
>
>Fri, May 06, 2005 at 09:11:13PM +0300, Ilkka Priha wrote:
>
>What kind of writer are you using and how long are the items? 66
>
>000 HTML
>
>table rows with 5 numeric columns consumes about 32 MB when
>
>buffered into a
>
>string writer. If the items are longer, it takes much more
>
>memory.
>
>Ilkka
>
>
>lawrence (AT) marchex (DOT) com wrote:
>
>I tried putting static HTML in the #foreach loop, same problem.
>
>Then I
>
>tried setting the large list as a "local" varible:
>
>set ($localList = $List)
>foreach($row in $localList)
>
>
>but that didnt help either. I don't see why velocity should
>
>use
>
>250+ MB
>
>of memory iterating over a li