Using completable futures in Java

This week I needed to download a lot of files on a remote server using multi-threading.
I always try to avoid multi-threading and treat it as an absolute last option as it makes code significantly more complicated as well as introducing all the really hard to debug bugs that it brings (race conditions, thread starvation\locks etc).
Unfortunately, as the bottleneck was the network, I couldn’t optimise to speed up the solution. So after looking for an alternative, I gave in and looked up how to do multi-threading in Java.

After some research, I decided that out of all the various multi-threading API’s, the Callable Future API was the one to go with. This is because the code that needs to process the downloaded files should not execute until all files are present. The Callable Future API handles this case the best.

The first thing I needed to do was find out how many files were in the remote directory. This was achieved by using an SSH library call JSch (

Once I ran a ls command, I read the returned input stream into a hash set of strings and then chunked the hash set by the amount of threads I want to download the files. Each chunk of file names are fed to each thread to which they go off and retrieve those files using a SCP command.
When all the threads have completed their downloading, the Join function collects the reults (which is a SshDownloadResult class that encapsulates sucessfully and unsuccessfully downloaded files).

Here is the code:

final ExecutorService executorService = Executors.newFixedThreadPool(threadsCount);

final List downloadTasks = IntStream
.range(0, threadsCount)
.map((threadIndex) -> (Callable) () -> {
final HashSet filesNamesForCurrentThread =
return download(connectionSettings, absoluteDownloadFromDirectory,
downloadToDirectory, filesNamesForCurrentThread);

final List futures = downloadTasks
.map(task -> CompletableFuture.supplyAsync(() -> {
try { return; }
catch (Exception e) {
log.error("Download thread " + task.toString() + " threw an exception.
Message = " + e.getMessage(), "runUsingSsh");
return null;    // Todo: Handle this better.
}, executorService))

final List downloadResult = futures