Benchmarking

From WatermillWiki

1 Dataset
2 Watermarking speed and rate
3 Attacks
4 ForestCover dataset
- 4.1 To test your installation
- 4.2 Tests

Dataset

Watermill features a built-in tool to benchmark your system. It uses a synthetic relation database B(n,p,k) storing products and sales, n being the number of products, p the number of carts (customers) and k the number of products per cart. Each cart is filled which k products, randomly chosen. The database is watermarked while respecting the constraints:

the distortion on the cost of each product does not exceed 1 Euro;
the distortion on the total cost for each shopping cart does not exceed 1 Euro.

[edit]

Watermarking speed and rate

Watermarking speed can be tested using the following command:

bench speed "poolName" 1000 10 3 50 2 1 3 "filePrefix";

poolName is either akp (optimized algorithm) or greedy (please be patient!).
1000 is the number of products.
10 is the minimum number of carts.
3 is the number of products per cart.
50 is the maximum number of carts.
2 is the additive increment of the number of carts.
1 is the multiplicative increment of the number of carts.
3 is the number of time each measurement is iterated.

The results are written to a data file which name is built by appending the parameters to the filePrefix. For instance, results of the previous experience will be stored in filePrefix-akp-1000-10-3-50-1-1-1.data. This file is self-descriptive and includes gnuplot plotting instructions.

[edit]

Attacks

Robustness against attacks can be evaluated using the following commands:

 bench attack "subset" "gamma=1:odir=outputdir/:iterations=1:plot=true" "filePrefix";

subset is the type of the attack (subset or epsilon). Full explanations has to be looked for in the source code.

[edit]

ForestCover dataset

Data can be downloaded from UCI KDD Archive. It consists of The forest cover type for 30 x 30 meter cells obtained from US Forest Service (USFS) Region 2 Resource Information System (RIS) data. It is used as way to evaluate watermarking speeds. It contains 581012 tuples.

[edit]

To test your installation

install and run rmill (the relation part of watermill), full instructions can be accessed from the main page.
Create a forestcover database in your favorite RDBMS.

mysqladmin create forestcover

Create the database schema (e.g. by using one of the following script files MySQL PostgreSQL).

mysql -u user -p forestcover < create-table.sql

install the covertype dataset onto the freshly created database, data can converted to sql statements by piping it through the following perl script.

cat covtype.data | tosql.pl | mysql -u user -p forestcover

Run the benchmark by executing the commands of this script (optimized/greedy).

rmill < fctconstraints-akp.rmill

[edit]

Tests

Revision	SGBD(rev.)	Strategy	CPU	Ram	OS(arch.)	Watermarking time	Comments
N/A	Optimized	Mysql (5.0.18)	P4HT@3.2GHz	1GB	Debian GNU/Linux(64bit)	8min03 (pool) + 4min57
N/A	Greedy	Mysql (5.0.18)	P4HT@3.2GHz	1GB	Debian GNU/Linux(64bit)	several days

Retrieved from "http://watermill.sourceforge.net/wiki/index.php/Benchmarking"

Benchmarking

From WatermillWiki

Contents

Dataset

Watermarking speed and rate

Attacks

ForestCover dataset

To test your installation

Tests

Views

Personal tools

Navigation

Search

Toolbox