I wish to add a section at The Data Compression News Blog (
www.c10n.info ) which can allow for easily monitoring any new research
papers related to data compression. How to best go about doing this?
arXiv and citeseer allow for automatic harvesting. Abstracts listings
of few IEEE journal's are also available in parsable formats. I can
then apply some heuristic filters to new submissions to these sites.
Will this be a large enough representation of research activity? What
all other prominent sources can be included? (ofcourse published papers
are themselves a small part of all research activity, but you know what
I mean.)
Any other advice on how to accomplish this, are there any other sources
which can make this job easier? or if it is even possible to have a
large enough coverage for such a service to be useful.
I think some of us must be already doing this, so must be having some
experience and information on information sources.
btw, I have further improved the filters applied to the automated
Newsbot at www.c10n.info/newsbot , it gets updated every hour. And I
have been exploring some options for monitoring all patent activity
(granted patents or applications filed) and this seems more possible
than research paper monitoring, any advice on these will also be
helpful.
Sachin Garg [India]
www.sachingarg.com | www.c10n.info

"Sachin Garg" <schngrg@gmail.comwrote in message
news:1158407464.783454.106370@
|
| I wish to add a section at The Data Compression News Blog (
| www.c10n.info ) which can allow for easily monitoring any new research
| papers related to data compression. How to best go about doing this?
|
| arXiv and citeseer allow for automatic harvesting. Abstracts listings
| of few IEEE journal's are also available in parsable formats. I can
| then apply some heuristic filters to new submissions to these sites.
| Will this be a large enough representation of research activity? What
| all other prominent sources can be included? (ofcourse published papers
| are themselves a small part of all research activity, but you know what
| I mean.)
|
| Any other advice on how to accomplish this, are there any other sources
| which can make this job easier? or if it is even possible to have a
| large enough coverage for such a service to be useful.
|
| I think some of us must be already doing this, so must be having some
| experience and information on information sources.
|
|
| btw, I have further improved the filters applied to the automated
| Newsbot at www.c10n.info/newsbot , it gets updated every hour. And I
| have been exploring some options for monitoring all patent activity
| (granted patents or applications filed) and this seems more possible
| than research paper monitoring, any advice on these will also be
| helpful.
|
| Sachin Garg [India]
| www.sachingarg.com | www.c10n.info

You are aware that GGLE has a SCHLAR SEARCH.

GGLE Scholar
http://scholar.google.com/
Search scholarly papers

Now the question is whether you can improve the situation or just offer
a different solution to the same result.

Data organization will be a major necessity as people have the resources
to create vast swaths of data, but rarely take the time to organize it
efficently much less reshuffle the whole mess later if new information
formats become useful. needs to be able to search data sets quickly,
organize in a number of generic classifaction subsets for optimal searching,
and then allow priority functions to sort the classified data sets for
usefulness and other human-based ratings along the lines of occassional
randomization in the top lists to toss up rarely looked at information
pointers in case the human has overlooked something important.

There are a bundle of ways unique, obscure, clever, and generic to
accomplish the results, but oftentimes these are basically a function set
overlapping the obvious generic function sets already commonly in usage.

At the core, many people prefer predictable over customizable.
So I would recommend offering the best functions first, then if the user
wants more of a custom search then allow them to do so. If the user allows,
it would be wise to use all of the searches if time permits, then toss up
the best solutions that you can find for the submitted questions.

Still, you'll have a hard time beating GGLE as they are rather clever
at what they do, and they've made money doing it.

George Johnson wrote:
"Sachin Garg" <schngrg@gmail.comwrote in message
news:1158407464.783454.106370@
|
| I wish to add a section at The Data Compression News Blog (
| www.c10n.info ) which can allow for easily monitoring any new research
| papers related to data compression. How to best go about doing this?
|
| arXiv and citeseer allow for automatic harvesting. Abstracts listings
| of few IEEE journal's are also available in parsable formats. I can
| then apply some heuristic filters to new submissions to these sites.
| Will this be a large enough representation of research activity? What
| all other prominent sources can be included? (ofcourse published papers
| are themselves a small part of all research activity, but you know what
| I mean.)
|
| Any other advice on how to accomplish this, are there any other sources
| which can make this job easier? or if it is even possible to have a
| large enough coverage for such a service to be useful.
|
| I think some of us must be already doing this, so must be having some
| experience and information on information sources.
|
|
| btw, I have further improved the filters applied to the automated
| Newsbot at www.c10n.info/newsbot , it gets updated every hour. And I
| have been exploring some options for monitoring all patent activity
| (granted patents or applications filed) and this seems more possible
| than research paper monitoring, any advice on these will also be
| helpful.
|
| Sachin Garg [India]
| www.sachingarg.com | www.c10n.info

You are aware that GGLE has a SCHLAR SEARCH.

GGLE Scholar
http://scholar.google.com/
Search scholarly papers

Now the question is whether you can improve the situation or just offer
a different solution to the same result.
[snip]

"Google Scholar" and "Windows Live Academic" are great search services
but AFAIK they don't provide what I want to have. I think a simple news
type service for research papers can be helpful for some of us.

This doesn't necessarily needs the complexities of managing large data
sets or of complicated ranking needed for effective search results. I
just need to gather data, filter it, and present it in a human
digestible format.

A 'compression related research paper monitoring service' can be built
using Google Scholar or Windows Live Academic, but unfortunately
currently none of these two provide APIs to use their data in the way
needed.

I probably could have 'stood on shoulders of giant' ;-) and built
something similar to what I have done using Google News at
www.c10n.info/newsbot but that will probably have to wait till they
open up their scholar service a bit more. Windows live search
theoretically could have been used, but parts of their site are broken,
probably due to some recent update. If they fix it soon, that can be
used too. Gathering all the information independently is also possible,
but would need more effort and the result might not be as
comprehensive.

Still, you'll have a hard time beating GGLE as they are rather clever
at what they do, and they've made money doing it.

Beating google isn't the intention, intention is to use the best of
available services to build what I think we can use but they don't give
(atleast not yet) - A regularly updated reverse chronological listing
of all domain specific papers.

Sachin Garg [India]
www.sachingarg.com | www.c10n.info