This
page contains experimental results showing the performance of LPG 1.0 in
the test problems of the 3rd International Planning Competition (IPC):
www.dur.ac.uk/d.p.long/competition.html.
Notice
that the results that we present here are not exactly the same as the official
results of the competition, where for lack of time we were not able to
run our system a sufficient number of times to obtain meaningful statistical
data. However, in general the new results are very similar to those of
the competition, with some considerable improvement in Satellite Complex
and in the Rovers domains, where many problems could not be solved due
to a bug in the parser of our planner that was easily fixed right after
the competition. Overall, the number of problems attempted
in the new tests by our planner was **468** (over a total of 508 problems),
and the success ratio was **94.4%** (the problems attempted by LPG
in the competition were 428 and the success ratio 87%), which by far the
highest success ratio among the fully-automated planners of the competition

All tests were
conducted on the official machine of the competition, an AMD Athlon(tm)
MP 1800+ (1500MHz) with 1 Gbytes of RAM.

We ran LPG with the same default settings for
every problem attempted: maximum numbers of search steps = 500; maximum
numbers of restarts for each run = 50; inconsistency selection strategy
= 22; noise factor: 0.1. The results
for LPG correspond to median values over five runs of LPG for each problem
considered. The CPU-time limit for each run was 5 minutes, after
which termination was forced (when the CPU-time limit was exceeded in 3
or 4 runs, instead of the median value, we considered the worst
of the remaining, successful runs. This happened in 15 of the 424
problems solved using WALKPLAN).

The performance of LPG was tested in terms of
both CPU-time required to find a solution (**LPG-speed**) and quality
of the best plan computed (**LPG-quality**). In all plots, on the x-axis
we have the problem names (simplified with numbers). On the y-axis, in
the plots regarding LPG-speed we have CPU-milliseconds (logarithmic scale),
while in the plots regarding LPG-quality we have the quality of the plans
computed measured using the plan metric specified for the corresponding
problem.

The plots showing the performance of LPG with
respect to all other fully-automated planners of the 3rd IPC can be seen
here:
** LPG
compared with all other planners**

In order to derive some general results about the performance of our approach with respect to all other planners of the competition, we have compared LPG with the best result over all the other planners. We will indicate these results as if they were produced by an hypothetical ``SuperPlanner''.

The plots showing the performance of LPG with
respect to the SuperPlanner can be seen here:
** LPG
compared with the SuperPlanner**

The (gzip'ed) tar file containing all solutions
found by LPG can be dowloaded *here*

AG, 18 Novembre 2002