Hello all
As part of my research under my professor I have to implement a web
interface to their benchmarking data.
PHP is the chosen web language but we are little worried about the
database. The benchmark data comes to us in XML format (e.g.
http://www.matf.bg.ac.yu/~).
We have to implement an interface to query them, get data, update etc.
We even can change schema in the form of attributes. . The data size
would be around 100 MB each XML with around 100 different XMLs.
The load would be max 5-10 users any given time, batch updates once a
month and heavy load probably 2-3 times a month. Mission criticality is
not important, we can get it down sometimes. Which db would you suggest?
I did Google research and as of now - I like eXist, Sedna (they seem to
have good PHP wrapper support) and Timber. Another thing would be good
documentation and support.
Any suggestions?
Ritesh

Hi

Some questions

As part of my research under my professor I have to implement a web
interface to their benchmarking data.

PHP is the chosen web language but we are little worried about the
database. The benchmark data comes to us in XML format (e.g.
http://www.matf.bg.ac.yu/~).
We have to implement an interface to query them, get data, update etc.

You can parse the XML, extract the data, put it to an SQL DB and move
the XML to /dev/null (delete it).
If you do that, you don't need an XML DB.
Is this possible?

We even can change schema in the form of attributes. . The data size
would be around 100 MB each XML with around 100 different XMLs.

What do you mean by "different XMLs"?
Are you looking for a maschine that makes SQL Tables from XML?
What is inside of the 100MB XML? Your example is a MathML Formula.

The load would be max 5-10 users any given time, batch updates once a
month and heavy load probably 2-3 times a month. Mission criticality is
not important, we can get it down sometimes. Which db would you suggest?

I did Google research and as of now - I like eXist, Sedna (they seem to
have good PHP wrapper support) and Timber. Another thing would be good
documentation and support.

With an XML DB you can query data using XPATH. Is that the thing you
want? supports that for example.

Bernhard

Hello

Bernhard Zwischenbrugger wrote:
Hi

Some questions

>As part of my research under my professor I have to implement a web
>interface to their benchmarking data.
>>
>PHP is the chosen web language but we are little worried about the
>database. The benchmark data comes to us in XML format (e.g.
>http://www.matf.bg.ac.yu/~).
>We have to implement an interface to query them, get data, update etc.

You can parse the XML, extract the data, put it to an SQL DB and move
the XML to /dev/null (delete it).
If you do that, you don't need an XML DB.
Is this possible?

No. My professor is dead against that. Many people have suggested me
doing that. Why? Are XML databases incorrectly implemented or are bad?

>We even can change schema in the form of attributes. . The data size
>would be around 100 MB each XML with around 100 different XMLs.

What do you mean by "different XMLs"?
Are you looking for a maschine that makes SQL Tables from XML?
What is inside of the 100MB XML? Your example is a MathML Formula.

By different XMLs I meants, different XML files. So we can have 40 XML
files with around 50-100 MB each.

Yes, they will have lot sof those MathML formulas. Its benchmarking data
from some theoritical group that my professor works with. They have all
their database in XML so relational database is not possible otherwise
we will have to convert them between XML and relational all the time.

>The load would be max 5-10 users any given time, batch updates once a
>month and heavy load probably 2-3 times a month. Mission criticality is
>not important, we can get it down sometimes. Which db would you suggest?
>>
>I did Google research and as of now - I like eXist, Sedna (they seem to
>have good PHP wrapper support) and Timber. Another thing would be good
>documentation and support.

With an XML DB you can query data using XPATH. Is that the thing you
want? supports that for example.

Yeah but looking at
, I could not
find whether its free. I might be wrong but the info is not easily found
there.

the contrary IBMs offering at
looks FREE.

The problem is that both the above two database are beasts in themselves
and I just require 10% of what they do :)

We looked into Berkeley DB also but their support for PHP is not that
great. We have compile the module by ourselves etc (this is not a
problem though as we have the technical know how to do that) but lot of
people have suggested that its not a very stable system. I have not done
any benchmarking on it but I dont want to change my underlying DB couple
of months down the line just because we found out its not stable.

Apart from the above big three, the other free and reasonable good
implementation seems to be eXist, Timber and Sedna. This I am just
saying by reading their website. I have not used them.

So my question is: out of the six systems listed above, which one you
would suggest? Has anybody used any of the above system before? What are
your experiences?

Bernhard

Ritesh

Hi again

I don't know what the DB should do for you.

If you simple want to select subtrees from your big XML, you can put
the XML Files to the filesystem and use XPointer for query.

Here an example:

<?php
$xml=<<<END
<result xmlns:xi="" xmlns:xhtml="http://www.w3.org/1999/xhtml">
<xi:include href="#xmlns()xpointer(//xhtml:div[1]//xhtml:div[1])" parse="xml">
<xi:fallback>
<error>not found</error>
</xi:fallback>
</xi:include>
</result>
END;

$dom=domDocument::loadXML($xml);
$dom->xinclude();

header("Content-type:text/xml");
echo $dom->saveXML();
?>

This example loads an XML (xhtml) from the web (filesystem is also
possible) and outputs a subtree of the document.
The subtree is selected by an XPointer expression.

The XML:
The XPointer:
"#xmlns()xpointer(//xhtml:div[1]//xhtml:div[1])"
(the namespace part is a little tricki)

To parse 100MByte takes some time.
A database can improve speed, the query syntax (XPointer) maybe is the
same.
Without knowing what the output of the database should be und the query
parameters, it is not easy to tell what db is the best for you.

A solution could be to split the 100MByte file in small parts and store
the subtrees to blobs in mySql. You can build an index based on XPath
and it could be a real fast solution.

Bernhard

Fri, January 26, 2007 2:36 pm, Ritesh Nadhani wrote:
Hello all

As part of my research under my professor I have to implement a web
interface to their benchmarking data.

PHP is the chosen web language but we are little worried about the
database. The benchmark data comes to us in XML format (e.g.
http://www.matf.bg.ac.yu/~).
We have to implement an interface to query them, get data, update etc.

We even can change schema in the form of attributes. . The data size
would be around 100 MB each XML with around 100 different XMLs.

The load would be max 5-10 users any given time, batch updates once a
month and heavy load probably 2-3 times a month. Mission criticality
is
not important, we can get it down sometimes. Which db would you
suggest?

I did Google research and as of now - I like eXist, Sedna (they seem
to
have good PHP wrapper support) and Timber. Another thing would be good
documentation and support.

Any suggestions?

MySQL with the XML Engine would be my first suggestion to check out.

You would just slurp down the XML and then use simple standard SQL
queries on the local copy.

I dunno how fast/stable it is, nor even what version of MySQL it
appears in, but it's there somewhere.

There are almost for sure other more common RDBMS that let you
"pretend" that XML is just another database storage file format, and
run regular old SQL queries on it. I think PostgreSQL can do this,
actually. I wouldn't be shocked to find out that SQL Server even has
something truly ugly to do it, but it would work.

An even simpler method, if you only get updates once a month, would be
to write a monthly "import" script that just loads in the XML to a
normal database, with some kind of automated conversion of the XML
attributes/tags to column names.

Fri, January 26, 2007 6:10 pm, Ritesh Nadhani wrote:
Hello

Bernhard Zwischenbrugger wrote:
>Hi
>>
>Some questions
>>
As part of my research under my professor I have to implement a web
interface to their benchmarking data.

PHP is the chosen web language but we are little worried about the
database. The benchmark data comes to us in XML format (e.g.
http://www.matf.bg.ac.yu/~).
We have to implement an interface to query them, get data, update
etc.
>>
>You can parse the XML, extract the data, put it to an SQL DB and
>move
>the XML to /dev/null (delete it).
>If you do that, you don't need an XML DB.
>Is this possible?
>>
>
No. My professor is dead against that. Many people have suggested me
doing that. Why? Are XML databases incorrectly implemented or are bad?

I think it's safe to say that at least some XML databases are
incorrectly implemented, or even bad. :-)

The answers you keep getting are because without a driving reason to
use the features specific to XML, using XML as the database backend is
probably a Bad Idea.

So unless you explain WHY you want the XML db, and unless those
reasons make sense to need an XML db, you'll keep getting people
telling you, effectively, to "Don't do that."

We even can change schema in the form of attributes. . The data
size
would be around 100 MB each XML with around 100 different XMLs.
>>
>What do you mean by "different XMLs"?
>Are you looking for a maschine that makes SQL Tables from XML?
>What is inside of the 100MB XML? Your example is a MathML Formula.
>>
>
By different XMLs I meants, different XML files. So we can have 40 XML
files with around 50-100 MB each.

Yes, they will have lot sof those MathML formulas. Its benchmarking
data
from some theoritical group that my professor works with. They have
all
their database in XML so relational database is not possible otherwise
we will have to convert them between XML and relational all the time.

How exactly will your converted XML get folded back into the other
data stores?

This is your big challenge, upon which everything else hinges.

The load would be max 5-10 users any given time, batch updates once
a
month and heavy load probably 2-3 times a month. Mission
criticality is
not important, we can get it down sometimes. Which db would you
suggest?

You have to define "heavy load" unless your max 5-10 *is* the "heavy
load" in which case you'd have to work hard to screw this up enough to
not be able to handle the load, I think

Though I dunno for sure what performance of XML backend is like

I did Google research and as of now - I like eXist, Sedna (they
seem to
have good PHP wrapper support) and Timber. Another thing would be
good
documentation and support.

I've never even heard of eXist nor Sedna, which is why I'm suggesting
you choose something a bit (okay a LT) more mainstream like MySQL,
where you'll have zillions of fellow users to help out.

So my question is: out of the six systems listed above, which one you
would suggest? Has anybody used any of the above system before? What
are
your experiences?

I've never actually been silly enough to insist on storying actual
data as XML and pretending it's a DB, rather than storing it as DB and
then writing scripts to output it as if it were XML

Unless you've got shared data segments scattered all across the world,
forcing it all into XML is probably the First Mistake. But since
that's your professors' problem, and not yours, I can see how you've
got little choice in the matter.

Thank you Bernhard and Richard

Your comments have been really helpful. I have a meeting with the
professor on Wednesday and I will lay down all this points in front of
him then.

1/29/07, Richard Lynch <ceo (AT) l-i-e (DOT) comwrote:
Fri, January 26, 2007 6:10 pm, Ritesh Nadhani wrote:
Hello

Bernhard Zwischenbrugger wrote:
>Hi
>>
>Some questions
>>
As part of my research under my professor I have to implement a web
interface to their benchmarking data.

PHP is the chosen web language but we are little worried about the
database. The benchmark data comes to us in XML format (e.g.
http://www.matf.bg.ac.yu/~).
We have to implement an interface to query them, get data, update
etc.
>>
>You can parse the XML, extract the data, put it to an SQL DB and
>move
>the XML to /dev/null (delete it).
>If you do that, you don't need an XML DB.
>Is this possible?
>>
>
No. My professor is dead against that. Many people have suggested me
doing that. Why? Are XML databases incorrectly implemented or are bad?

I think it's safe to say that at least some XML databases are
incorrectly implemented, or even bad. :-)

The answers you keep getting are because without a driving reason to
use the features specific to XML, using XML as the database backend is
probably a Bad Idea.

So unless you explain WHY you want the XML db, and unless those
reasons make sense to need an XML db, you'll keep getting people
telling you, effectively, to "Don't do that."

We even can change schema in the form of attributes. . The data
size
would be around 100 MB each XML with around 100 different XMLs.
>>
>What do you mean by "different XMLs"?
>Are you looking for a maschine that makes SQL Tables from XML?
>What is inside of the 100MB XML? Your example is a MathML Formula.
>>
>
By different XMLs I meants, different XML files. So we can have 40 XML
files with around 50-100 MB each.

Yes, they will have lot sof those MathML formulas. Its benchmarking
data
from some theoritical group that my professor works with. They have
all
their database in XML so relational database is not possible otherwise
we will have to convert them between XML and relational all the time.

How exactly will your converted XML get folded back into the other
data stores?

This is your big challenge, upon which everything else hinges.

The load would be max 5-10 users any given time, batch updates once
a
month and heavy load probably 2-3 times a month. Mission
criticality is
not important, we can get it down sometimes. Which db would you
suggest?

You have to define "heavy load" unless your max 5-10 *is* the "heavy
load" in which case you'd have to work hard to screw this up enough to
not be able to handle the load, I think

Though I dunno for sure what performance of XML backend is like

I did Google research and as of now - I like eXist, Sedna (they
seem to
have good PHP wrapper support) and Timber. Another thing would be
good
documentation and support.

I've never even heard of eXist nor Sedna, which is why I'm suggesting
you choose something a bit (okay a LT) more mainstream like MySQL,
where you'll have zillions of fellow users to help out.

So my question is: out of the six systems listed above, which one you
would suggest? Has anybody used any of the above system before? What
are
your experiences?

I've never actually been silly enough to insist on storying actual
data as XML and pretending it's a DB, rather than storing it as DB and
then writing scripts to output it as if it were XML

Unless you've got shared data segments scattered all across the world,
forcing it all into XML is probably the First Mistake. But since
that's your professors' problem, and not yours, I can see how you've
got little choice in the matter.