In this post I’ll show how to use the ExecutorService of the java.concurrent package, in order to start as many inkscape shells as processors available on the current machine and to distribute a whole bunch of conversion tasks wisely on the cores. On my quad core I got a speedup of about 3.5, which is really near to 4.
If you are not familiar with Java facilities like Thread, ThreadLocal, Callable<V>, and anonymous classes this post will not tell you how they work in depth but might give you an idea. For further reading I suggest a book on advanced Java features.
At first lets have a look at the main method of our SVGExporter. It reads in the with and height parameter as well as the output format:
public static void main(String[] args) throws InterruptedException, ExecutionException {
if(args.length == 0) {
System.out.println("SVG-Exporter\n" +
"Parameter: --with <number> --height <number> --exportFormat [ png | ps | eps | pdf | plain-svg ]\n" +
" All parameters are optional. If no width or height parameter is given, the corresponding values of \n" +
" the source document will be used. The default export format is \"png\".\n" +
"\n" +
"Have a lot of fun!");
}
long startTime = System.currentTimeMillis();
// Read arguments
final List<File> files = new ArrayList<File>();
Integer width = null, height = null;
String exportFormat = null;
String currArg = null;
for(final String arg : args) {
if(arg.startsWith("--")) {
currArg = arg.substring(2);
if( !(
currArg.equalsIgnoreCase("width") ||
currArg.equalsIgnoreCase("height") ||
currArg.equalsIgnoreCase("exportFormat")
)) {
throw new IllegalArgumentException("unknown argument: \"" + currArg + "\"");
}
continue;
}
if(currArg != null) {
if(currArg.equalsIgnoreCase("width")) {
try {
width = Integer.parseInt(arg);
}
catch(NumberFormatException e) {
throw new IllegalArgumentException("width argument is not an integer", e);
}
}
if(currArg.equalsIgnoreCase("height")) {
try {
height = Integer.parseInt(arg);
}
catch(NumberFormatException e) {
throw new IllegalArgumentException("height argument is not an integer", e);
}
}
if(currArg.equalsIgnoreCase("exportFormat")) {
exportFormat = arg;
}
currArg = null;
continue;
}
final File file = new File(arg);
if(file.canRead() && file.isFile()) {
files.add(file);
}
else {
throw new IllegalArgumentException("Given file \"" + file.getAbsolutePath() + "\" is not valid. Properties: \n" +
"exists: " + file.exists() + "\n" +
"can read: " + file.canRead() + "\n" +
"is file: " + file.isFile());
}
}
// Start export
new SVGExporter(exportFormat, width, height).convertWithInkscape(files);
System.out.println("zeit: " + (System.currentTimeMillis() - startTime));
}
Next, let’s have a look at the convert method convertWithInkscape. First, it iterates over the conversion tasks and creates Callable objects for concurrent execution of these tasks. Every callable has a method call() which starts the conversion of one file. Second, an executor service is created having a thread pool with as many threads as processors are available to the JVM. Third, the tasks are handed over to the executor service, in order to be processed concurrently. Fourth, the algorithm iterates over all tasks and waits for their termination and checks whether the created file exists. As many tasks as inkscape shell processes have been opened are started, terminating them after all conversion tasks have been finished successfully. Finally, the executor service is shut down:
public void convertWithInkscape(Collection<File> sourceFiles) throws InterruptedException,
ExecutionException {
// Create a collection for the export tasks.
final Collection<Callable<File>> tasks = new ArrayList<Callable<File>>(
sourceFiles.size());
// Iterate through the source files and create export tasks
for (final File sourceFile : sourceFiles) {
tasks.add(new Callable<File>() {
// an export task
});
}
final int availableProcessors = Runtime
.getRuntime().availableProcessors();
System.out.println("available processors: " + availableProcessors);
// Create a thread pool for the tasks.
final ExecutorService pool = Executors.newFixedThreadPool(availableProcessors);
// Execute the tasks concurrently and wait for execution to finish.
for (final Future<File> future : pool.invokeAll(tasks)) {
final File targetFile = future.get();
// Check whether the target file exists.
if (!targetFile.exists()) {
System.err.println("target file \"" + targetFile.getAbsolutePath() + "\" does not exists.");
}
}
final Collection<Callable<Process>> closeTasks = new ArrayList<Callable<Process>>(
inkscapeShells.size());
// Close shells.
for (final Process shell : inkscapeShells) {
closeTasks.add(new Callable<Process>() {
// a task for terminating the inkscape shells
});
}
// Wait for closes to finish.
for (final Future<Process> future : pool.invokeAll(closeTasks)) {
final Process process = future.get();
try {
if (process.exitValue() != 0) {
System.err.println("Return code: " +
process.exitValue());
}
}
catch(IllegalThreadStateException e) {
System.err.println("shell process did not finish: " + e.getMessage());
}
}
pool.shutdown();
}
private File source2targetFile(File svgFile) {
final String fileName = svgFile.getName();
if(fileName.length() < 5 ||
!fileName.substring(fileName.length()-4).equalsIgnoreCase(".svg")) {
throw new IllegalArgumentException("file is not an svg file");
}
return new File(svgFile.getAbsolutePath().substring(0, svgFile.getAbsolutePath().length()-4) + "." + exportFormat);
}
The callables have been omitted from the code above. The first is responsible for starting a conversion on an inkscape shell. For every thread in the thread pool a new inkscape shell will be created if not existent, stored to a ThreadLocal object so that it can be accessed from inside this very Thread and will be reused as additional tasks are executed on it. So, the task retrieves the current shell first, starts the conversion by typing the respective commando into the inkscape shell and checks for the success of the operation afterwards:
@Override
public File call() throws Exception {
final Process process = retrieveShell();
final File targetFile = export(sourceFile, process);
final BufferedReader reader = retrieveReader(process);
checkExportSuccess(sourceFile, targetFile, reader);
return targetFile;
}
private final Process retrieveShell() throws IOException {
Process process = null;
if(SVGExporter.shell.get() != null) {
final Process currShell = SVGExporter.shell.get();
try {
currShell.exitValue();
// create new shell.
process = createNewShell();
}
catch(IllegalThreadStateException e) {
// normal case
process = currShell;
}
}
else {
process = createNewShell();
}
return process;
}
private final Process createNewShell() throws IOException {
final Process process = Runtime.getRuntime().exec("inkscape --shell");
inkscapeShells.add(process);
SVGExporter.shell.set(process);
return process;
}
private final BufferedReader retrieveReader(Process shell) {
final BufferedReader reader =
new BufferedReader(new InputStreamReader(shell.getInputStream()));
return reader;
}
private final File export(final File sourceFile, Process shell) {
final File targetFile = source2targetFile(sourceFile);
final PrintWriter writer = new PrintWriter(shell.getOutputStream());
writer.println(sourceFile.getAbsolutePath() + " --export-" + exportFormat + "=" +
targetFile.getAbsolutePath() +
(width != null ? " --export-width=" + width : "") +
(height != null ? " --export-height=" + height : ""));
writer.flush();
return targetFile;
}
private final void checkExportSuccess(final File sourceFile, final File targetFile,
final BufferedReader reader) throws IOException {
char character;
boolean success = false;
boolean beginLine = true;
while((character = (char)reader.read()) != -1) {
if(beginLine && character == '>') {
success = true;
break;
}
if(character == '\r' || character == '\n') {
beginLine = true;
}
else {
beginLine = false;
}
}
if(!success) {
System.err.println("Icon \"" + sourceFile
+ "\" couldn't be converted to \"" + targetFile + "\" with targetFormat: \"" + exportFormat +
(width != null ? ", width: " + width : "") +
(height != null ? ", height: " + height : ""));
}
}
As you can see, the anonymous inner class accesses fields (e.g. with, height, export format) and methods (source2targetFile()) of the outer class SVGExporter. This is one of the nice features of inner classes. The second callable terminates the passed inkscape shell:
@Override
public Process call() throws Exception {
quitShell(shell);
checkShellDestroyed(shell);
return shell;
}
private final void quitShell(Process shell) {
final PrintWriter writer = new PrintWriter(shell.getOutputStream());
writer.println("quit");
writer.flush();
}
private final void checkShellDestroyed(Process shell)
throws InterruptedException {
for (int i = 0; i < 10; i++) {
boolean finished = true;
try {
shell.exitValue();
}
catch (IllegalThreadStateException e) {
finished = false;
}
if (finished) {
return;
}
if (i < 10) {
Thread.sleep(100);
}
}
shell.destroy();
}
The complete sourcecode can be downloaded as an archive (svgexporter.zip).