Index
Home
FAQ
Run Rules
Building OSDB
Style Guide
Project Status
OSDB Links
-Get OSDB here!
-The CVS
Repository has the latest (maybe broken) sources
-OSDB mailing
lists
-Contact us
Other Links
-Linux Benchmark Suite
- Association Française des Utilisateurs d'Unix
et des Systèmes Ouverts
SSBA benchmark suite
OSDB has tests underway for
Informix
MySQL
PostgreSQL
OSDB needs tests for
FirebirdSQL
InterBase
Oracle
Sybase
ODBC
Others?
|
Welcome to the Open Source Database Benchmark web site!
To date, there has been no easy way to benchmark the performance of a
database system. The choices were
- Hire a consulting group specializing in benchmarks
- Purchase the rights to one of the important SPEC or TPC
benchmarks, and staff a group to implement and run the tests
- Review the results of published benchmarks and extrapolate them
to your situation
- Create your own benchmark suite
The cost of the first two options is well-justified in some cases.
If one is about to spend ten million dollars on computer
infrastructure (in addition to the cost of staffing, developing, and
maintaining the application for which the project is being
undertaken), spending one million dollars to confirm that you are
about to make the right choice is very sensible.
Option 3 is a good choice if you are quite sure that the published
benchmarks are relevant for your situation, and that you have the
expertise to tune the systems as well as the vendors who prepared
their systems for the benchmarks.
Note that many database vendors use licenses which prohibit the
public disclosure of database performance information without their
prior approval. Is this because of the difficulty of tuning their
systems correctly, or fear that real-world performance will vary from
their published benchmarks?
Option 4 is great if you have the time, resources, and expertise to
create a benchmark test suite.
If you lack the time and resources, or if you lack confidence in your
ability to construct a robust, reliable, repeatable benchmark, what
can you do?
OSDB was created to provide a solid, peer-reviewed
foundation for people of sufficient technical skill
to get a head start in running their own analysis of database and
system performance, and to share the results of the
labor (and joy!) in preparing this code.
Peer review is an important attribute of an open source
project. The leader of the OSDB project is a software practitioner of
varying skill in the arts of coding, databases, and benchmarks. By
pooling his efforts with those of others, OSDB should become the
robust, reliable, repeatable benchmark that it can be!
Sharing the results
In describing his invention, the "Franklin" stove, Franklin wrote:
"Gov'r. Thomas was so pleas'd with the construction of this stove, as
described in it, that he offered to give me a patent for the sole vending of
them for a term of years; but I declin'd it from a principle which has ever
weighed with me on such occasions, viz., That, as we enjoy great advantages
from the inventions of others, we should be glad of an opportunity to serve
others by any invention of ours; and this we should do freely and
generously."
From
The Autobiography of Benjamin Franklin, Part VI at
www.wakeamerica.com
None of this work would have been possible without the nearly infinite
number of inventions and ideas on which it is built. In that spirit,
we hope to contribute in some small way to the global pool of
intellectual wealth.
Headlines
- 30 Nov 2009
New direction!
As shared with the mailing lists earlier this month, the OSDB project has a new maintainer. My name is William Bartig, and I am excited to continue to move OSDB
towards the future. Many thanks go to Andy for his work with OSDB for all of these years, and I certainly hope he is able to find time to check on us now and then.
I have many plans for OSDB as we move forward, but for now, I will quote the first three goals that I expressed to the mailing lists:
1. I will be updating the Oracle test to fix some problems it has, including hard-coded values and a problematic multi-user test.
2. Port OSDB to an OO model in C++, which will make it much easier to develop tests. During the port, I will also work on the build system to make that a little easier to use and consistent between each type of test.
3. After the first two goals are met, I plan to spread the word about OSDB through various OSS channels, which I hope will bring more interest (and more contributors) to the project.
I welcome any kind of input on the project, and I look forward to working with everyone to bring an improved OSDB to more people who could benefit from it.
- 19 Oct 2004
OSDB, the Open Source Database Benchmark, version 0.17 has been released!
Featuring ports to DataBlitz and Oracle, as well as a preliminary Perl implementation, this is the
best iteration of OSDB yet!
This version of OSDB can
- Skip tests that are inappropriate for the database engine, with the --restrict switch
- Run under MPI, for true cluster and multiprocessor benchmarks
New results have been posted in the expected/ directory, suggesting useful ways of invoking OSDB.
Barring results to the contrary in the next couple of weeks, this will soon be the new "stable"
version of OSDB.
- 20 May 2003
Preview OSDB x0.15-1
The osdb-unstable x0.15-1 kit, available from the SourceForge site, introduces "dbuser"
and "dbpassword" options, allowing one to specify the database username and password.
- 11 May 2003
Preview the next version of OSDB
The osdb-unstable x0.15-0 kit, available from the SourceForge site, includes the ability
to generate your own data to test databases from 4MB to 400GB.
This kit lacks Informix functionality, and the -ui interface tests. As always, let us
know of any problems.
- 13 March 2002
Improved Support for MySQL and PostgreSQL Benchmarks
The "Unlucky release" of OSDB, version 0.13, was quickly
replaced with version 0.14 which
introduces support for database product-specific options.
Specifically, for
MySQL, we now support the InnoDB and BDB engines
via the
--mysql={innodb | bdb}
option.
For you long-suffering OSDB PostgreSQL users, we
offer
--postgresql=no_hash_index
to work around the hash index problems of OSDB with PostgreSQL
V7.1 and 7.2.
As always, let us know of any problems.
May the source be with you!
- 20 February 2002
OSDB version 0.12 releases with improved portability and support for MySQL 3.23, PostgreSQL
7.2
OSDB version 0.12 features improved portability to most *nix systems, and a new build procedure
that lets one specify which of several possible database installations should be installed.
- 15 February 2002
OSDB joins the SourceForge
DataBase Foundry!
- 30 September 2001
OSDB version 0.11-- The silent release
we forgot to tell anyone about it!
OSDB version 0.11 features all new documentation, sample output, corrections to some of the SQL
code, and a script that makes it possible to generate your own data sets (rather than downloading
our sample data.
- 19 April 2001
OSDB version 0.10 released with pretty good support for Informix®, MySQL®, and
PostgreSQL®
This version has pretty good support for:
Informix® |
Embedded SQL (version 2.95.2) |
MySQL® |
Native API (version 3.23.32) |
PostgreSQL® |
Native API, Embedded SQL (version 7.0.3) |
In addition, this version supports tests via the user
interface programs (dbaccess, mysql, psql), but these
are not as reliable as the embedded SQL or native API
tests.
|