I understandbecause I've experienced it. I think the answer is to
'parallelize' the docFreq processand or try to make use of the
docFreq(Terms[]). By passing an Array of Terms, you can avoid the 'call
per Term' per remote and just make a single docFreq call per remote.
You might have to extend the ParallelMultiSearcher and create a threaded
docFreq method.
Message
From: Yura Smolsky [mailto:info (AT) altervisionmedia (DOT) com]
Sent: Friday, September 15, 2006 8:54 AM
To: java-user (AT) lucene (DOT) apache.org
Subject: Re: ParallelMultiSearcher and docFreq
Hello, Yura.
Does anyone understand my email? Maybe my English is too bad
Thanks.
YSHere is the situation. I have ParallelMultiSearcher object
YSinitializated with two or more RemoteSearchable's.
YSI run PrefixQuery search on some keyword field, say "link". When I
run
YSsearch starting just with letter "w" (link:w*) then I should have
like 5k
YSresults.
YSAs I know when I perform search on ParallelMultiSearcher query is
YSbeing rewritten at first. So my prefix search is being rewritten
with
YS"link:wordlist.com link:web.com and so on about 2-3k of terms". Then
as I
YSunderstand from debugging for each such term
YSParallelMultiSearcher performs docFreq
YSrequests to RemoteSearchables (2-3k calls). So we have many requests
YSto docFreq method and these operations take like 95% of all search
time.
YSI see that we have docFreqs method for RemoteSearchable, but it has
YSnot being used.
YSIs there any way to get rid of those multiple calls of docFreq?

Hello, Ronald.

What I have found that nothing except createWeight uses that
docFreqs(Term[]) method
Maybe I need to parallelize it But I dont understand something.

When does Multisearcher.createWeight() is being called, b/c only this method
used docFreqs and this method creates HashMap of docFreqs of terms? Is
this method is being user for rewrite of query inside of
ParallelMultiSearher?

Also this method calls docFreqs of RemoteSearchables, I should be
receiving calls of docFreqs(Term[]) to the RemoteSearchable objects,
but I do not. Can somebody explain this?

And from which place am I receive those multiple calls of docFreq
method?

Thanks.

HRCLDI understandbecause I've experienced it. I think the answer is to
HRCLD'parallelize' the docFreq processand or try to make use of the
HRCLDdocFreq(Terms[]). By passing an Array of Terms, you can avoid the 'call
HRCLDper Term' per remote and just make a single docFreq call per remote.

HRCLDYou might have to extend the ParallelMultiSearcher and create a threaded
HRCLDdocFreq method.