I have a database with 2.5 million records (101 MB) storing a product SKU#
with a detailed description of the product. When searching for the SKU#,
the NT server crashes because of the database size. Can anyone recommend
another solution in addition to adding more RAM? A full file is sent to us
weekly but incremental daily updates are also available.
We are using WebCatalog 3.0 on Windows NT. I've read through the archive
talk lists & technical reference but didn't see any other options except
adding RAM.
I had a couple of ideas and would appreciate any recommendations or solutions.
1. Break down the database and write a file for every unique SKU #.
Use the Include tag to read in each file as needed. Problems:
Figuring out how to 'automate' the creation of multiple, separate sku files
on a weekly basis. Determining how to address daily updates received.
2. Split the single database into smaller databases sorted by SKU #.
However, I couldn't figure out the correct WebCatalog syntax for showif for
a range of numbers. Ex: SKU # is between 100000 - 300000, search through
databaseA. SKU# is between 300001 - 500000, search through databaseB.
Problems: I heard this solution doesn't address the problem because ALL
the files could potentially be loaded into cache and still crash our system due to
multiple users searching different SKUs. Verify?
3. Create a hybrid with WebCatalog and ODBC with SQL on the backend.
Would this address my cache/performance problem? If so, where can I find
more information?
I am new to WebCatalog so please let me know if there are other resources I
can use to research this problem. Or if this is not possible, please let me
know so I'm not spinning my wheels. Thank you for any help or suggestions!
Juliana Sun
|
I have a database with 2.5 million records (101 MB) storing a product SKU#
with a detailed description of the product. When searching for the SKU#,
the NT server crashes because of the database size. Can anyone recommend
another solution in addition to adding more RAM? A full file is sent to us
weekly but incremental daily updates are also available.
We are using WebCatalog 3.0 on Windows NT. I've read through the archive
Talk Lists & technical reference but didn't see any other options except
adding RAM.
I had a couple of ideas and would appreciate any recommendations or solutions.
1. Break down the database and write a file for every unique SKU #.
Use the Include tag to read in each file as needed. Problems:
Figuring out how to 'automate' the creation of multiple, separate sku files
on a weekly basis. Determining how to address daily updates received.
2. Split the single database into smaller databases sorted by SKU #.
However, I couldn't figure out the correct WebCatalog syntax for showif for
a range of numbers. Ex: SKU # is between 100000 - 300000, search through
databaseA. SKU# is between 300001 - 500000, search through databaseB.
Problems: I heard this solution doesn't address the problem because ALL
the files could potentially be loaded into cache and still crash our system due to
multiple users searching different SKUs. Verify?
3. Create a hybrid with WebCatalog and ODBC with SQL on the backend.
Would this address my cache/performance problem? If so, where can I find
more information?
I am new to WebCatalog so please let me know if there are other resources I
can use to research this problem. Or if this is not possible, please let me
know so I'm not spinning my wheels. Thank you for any help or suggestions!
Juliana Sun
DOWNLOAD WEBDNA NOW!
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...