NAVIGATION
CATEGORIES
REFERRENCE
LINKS
reg:blob data reading
13 answers - 539 bytes -
I think too, if they are Big-Blobs, it is better to store only a Reference
to a File.
May i know litlle more clearly about this?What it mean actually?
Right now what i am doing is, I have a directory in which somany files are
there and i have loaded all the files into database and the data(ie content
of files as blob).
But my application is reading only 512 bytes at a timeIf that is the case
how can i perform / handle this data .
Please suggest me your views.
Thanks a lot
Sandhya RNo.1 | | 495 bytes |
| 
9/18/06, sandhya <sandhyar (AT) amiindia (DOT) co.inwrote:
I think too, if they are Big-Blobs, it is better to store only a Reference
to a File.
May i know litlle more clearly about this?What it mean actually?
Store the path to the file in the database ( C:\somefile.dat or
/tmp/somefile.dat ).
Then open the file using regular file handing routines ( fopen() etc ).
To unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) orgNo.2 | | 1173 bytes |
| You mean,
I have to get the entire BLB from the DB and has to store it in some
temp file and reading it?
Is there no way we can read the required no.of bytes of the data from the
DB directly and setting the pointer or handle to the current position?
Please tell me whether the way i am thinking is wrong??
Help me
Thank you
Sandhya
Message
From: "Jay Sprenkle" <jsprenkle (AT) gmail (DOT) com>
To: <sqlite-users (AT) sqlite (DOT) org>
Sent: Monday, September 18, 2006 6:53 PM
Subject: Re: [sqlite] reg:blob data reading
9/18/06, sandhya <sandhyar (AT) amiindia (DOT) co.inwrote:
I think too, if they are Big-Blobs, it is better to store only a
Reference
to a File.
May i know litlle more clearly about this?What it mean actually?
Store the path to the file in the database ( C:\somefile.dat or
/tmp/somefile.dat ).
Then open the file using regular file handing routines ( fopen() etc ).
To unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) org
To unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) orgNo.3 | | 1166 bytes |
| Hello Jay,
Monday, September 18, 2006, 9:23:27 AM, you wrote:
JS9/18/06, sandhya <sandhyar (AT) amiindia (DOT) co.inwrote:
>I think too, if they are Big-Blobs, it is better to store only a Reference
>to a File.
>>
>May i know litlle more clearly about this?What it mean actually?
JSStore the path to the file in the database ( C:\somefile.dat or
JS/tmp/somefile.dat ).
JSThen open the file using regular file handing routines ( fopen() etc ).
JS
JSTo unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) org
JS
The whole reason I store files in the DB in the first place is to have
a single "package" to move around and backup when needed. My
application is storing whole series of PNG and JPG files in the
DB with meta data describing where the images came from.
I like the concept of being able to set an upper limit on the number
of retrieved bytes on a blob. I don't see any easy way to do it
though. My images files tend to be from 50-500K so, performance wise it's
pretty quick.No.4 | | 145 bytes |
| SQLite does not (at this time) have the ability to incrementally
read or write BLBs. You have to read and write the whole blob
all at once.No.5 | | 700 bytes |
| 9/18/06, Teg <Teg (AT) djii (DOT) comwrote:
Hello Jay,
The whole reason I store files in the DB in the first place is to have
a single "package" to move around and backup when needed. My
application is storing whole series of PNG and JPG files in the
DB with meta data describing where the images came from.
My technique won't help you then. I use it for things like scanning images
of documents and using the database to keep track of the documents.
I never have to search a picture using a select statement so it would
be silly for me to put them into the database. I just back up the file
system using off the shelf backup software and it works fine.No.6 | | 691 bytes |
| Thank you very much just i need confirmation on this
If you don't mind could you please tell me if i want to perform that kind of
operation Is there any way other than storing in some temp file and
readingusing normal fopen() calls.
Please do needful
Thanks a lot
Sandhya R
Message
From: <drh (AT) hwaci (DOT) com>
To: <sqlite-users (AT) sqlite (DOT) org>; "Teg" <Teg (AT) djii (DOT) com>
Sent: Monday, September 18, 2006 7:32 PM
Subject: Re: [sqlite] reg:blob data reading
SQLite does not (at this time) have the ability to incrementally
read or write BLBs. You have to read and write the whole blob
all at once.No.7 | | 1889 bytes |
| Hello Jay,
Monday, September 18, 2006, 10:05:19 AM, you wrote:
JS9/18/06, Teg <Teg (AT) djii (DOT) comwrote:
>Hello Jay,
>>
>The whole reason I store files in the DB in the first place is to have
>a single "package" to move around and backup when needed. My
>application is storing whole series of PNG and JPG files in the
>DB with meta data describing where the images came from.
JSMy technique won't help you then. I use it for things like scanning images
JSof documents and using the database to keep track of the documents.
JSI never have to search a picture using a select statement so it would
JSbe silly for me to put them into the database. I just back up the file
JSsystem using off the shelf backup software and it works fine.
JS--
JSSqliteImporter and SqliteReplicator: Command line utilities for Sqlite
JShttp://www.reddawn.net/~jsprenkl/Sqlite
JSCthulhu Bucks!
JShttp://www.cthulhubucks.com
JS
JSTo unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) org
JS
There's no right or wrong way. I was just suggesting that there are
cases where you want to store the whole file in the DB. I have an
application that generates 1000's of 150K compressed files. I've been
toying with the idea of shoving them all onto a DB because of the way
Windows groans when you have to enumerate folders with many small
files.
In the case of these small files, performance is dominated by
enumerating and decompressing so, even if it's a bit slower selecting
the files out of the DB, any improvement in enumeration speed would make
a noticeable performance boost.
The downside of course is I'd have to vacuum the tables from time to
time.No.8 | | 984 bytes |
| 9/18/06, Teg <Teg (AT) djii (DOT) comwrote:
Hello Jay,
There's no right or wrong way. I was just suggesting that there are
cases where you want to store the whole file in the DB. I have an
application that generates 1000's of 150K compressed files. I've been
toying with the idea of shoving them all onto a DB because of the way
Windows groans when you have to enumerate folders with many small
files.
The scanner I use takes care of that by making a directory for each
batch of documents scanned. I've seen that same thing on Windows
systems for other projects. I ended up having to create a manager that would
store files into numbered sub directories to avoid it.
The downside of course is I'd have to vacuum the tables from time to
time.
That's why we get paid the 'big bucks' ;)
To unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) orgNo.9 | | 1336 bytes |
| Jay Sprenkle wrote:
9/18/06, Teg <Teg (AT) djii (DOT) comwrote:
>Hello Jay,
>>
>The whole reason I store files in the DB in the first place is to have
>a single "package" to move around and backup when needed. My
>application is storing whole series of PNG and JPG files in the
>DB with meta data describing where the images came from.
>
My technique won't help you then. I use it for things like scanning
images
of documents and using the database to keep track of the documents.
I never have to search a picture using a select statement so it would
be silly for me to put them into the database. I just back up the file
system using off the shelf backup software and it works fine.
Everyone has different needs. We like keeping all of the data (blobs
included) in one data file. We also use the encryption extension, and
it is mandatory that our blobs be encrypted. I don't need "read
arbitrary byte ranges from a blob" for my work project, but I could use
them in a personal project that involves sqlite (no encryption here; but
it is important to keep all data in one data file).
To unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) orgNo.10 | | 964 bytes |
| Hello Dennis,
Monday, September 18, 2006, 11:50:03 AM, you wrote:
DJJay Sprenkle wrote:
>9/18/06, Teg <Teg (AT) djii (DOT) comwrote:
Hello Jay,
DJEveryone has different needs. We like keeping all of the data (blobs
DJincluded) in one data file. We also use the encryption extension, and
DJit is mandatory that our blobs be encrypted. I don't need "read
DJarbitrary byte ranges from a blob" for my work project, but I could use
DJthem in a personal project that involves sqlite (no encryption here; but
DJit is important to keep all data in one data file).
DJ
DJTo unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) org
DJ
I'm, probably going to be offering optional encryption too. Why did
you chose to use the SQLite encryption extensions versus just
encrypting the blobs after you read them back in and before you write
them out?No.11 | | 462 bytes |
| Mon, 18 Sep 2006 10:20:36 -0400, you wrote:
>There's no right or wrong way
There is viewpoint from June 2006 with title:
"To BLB or Not To BLB:
Large Storage in a Database or a Filesystem?"
at
%20report&id=1089
Maybe it turns a light on ;-)
Best Regards
Thomas
www.thlu.de
To unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) orgNo.12 | | 733 bytes |
| Teg wrote:
Hello Dennis,
I'm, probably going to be offering optional encryption too. Why did
you chose to use the SQLite encryption extensions versus just
encrypting the blobs after you read them back in and before you write
them out?
1) We wanted the entire database encrypted. There is sensitive non-blob
data too.
2) Dr. Hipp's encryption extension is well tested and already integrated
into sqlite.
3) The encryption is very transparent to the rest of our application. I
don't have to manually call functions to look up keys and encrypt or
decrypt blocks of data.
To unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) orgNo.13 | | 1042 bytes |
| Dennis Jenkins wrote:
Teg wrote:
>Hello Dennis,
>I'm, probably going to be offering optional encryption too. Why did
>you chose to use the SQLite encryption extensions versus just
>encrypting the blobs after you read them back in and before you write
>them out?
>
>
1) We wanted the entire database encrypted. There is sensitive
non-blob data too.
2) Dr. Hipp's encryption extension is well tested and already
integrated into sqlite.
3) The encryption is very transparent to the rest of our application.
I don't have to manually call functions to look up keys and encrypt or
decrypt blocks of data.
4) Updates to the blobs can now take advantage of the ACIDity of the
sqlite engine. (Is that a valid use of the acronym 'ACID'? Gotta love
the English language. We can conjugate anything anyway we want to.)
To unsubscribe, send email to sqlite-users-unsubscribe (AT) sqlite (DOT) org
QUESTION ON "Databases"
- test
- Need special sequence generator
- CLUSTERing on Insert
- FileMakerPro to postgreSQL
- blocking on insert
- FileMakerPro to postgreSQL
- blocking on insert
- sqlite_exec query string maximum length?
- Multiuser on LAN, how good/bad is it ?
- PostgreSQL slammed by PHP creator
- What is the Best Postgresql Load Balancing Solution available ?
- transaction confusion
- how sqlite works?
- Instructions For Building On Windows?
- Problem using loadable extension mechanism.
- Migration
- Which API to use to get resultant of query
- Which API to use to get resultant of query
- ECPG: non-integer constant in group by
- Error: database is locked on _all_ sqlite tables