Managed File Toolbox

Managed File Toolbox

I/O operations are many times considered boundary, less important, operations, however they are most frequently used, in an almost omnipresent way in all applications and in more that one flavour. Web applications, for example, often must cope with file download/upload features that are copied to a local filesystem. Commonly these features are introduced with the support of a third party API, and the real work boils down to stream plumbing.

The Managed File Toolbox is an essential MiddleHeaven Toolbox. Many other toolboxes use Managed File Toolbox services to handle file-like data and streams in a transparent way. We will se details on how when we talk about each one of them. Where we will look at Managed File Toolbox's simple core concepts and types.

Java allows for OS file system manipulation as we see fit in a very platform independent way, specially by means of java.io.File class. However Jar/Zip files that also may contain files and file groups like their where folders have a very different treatment. If you think of jar/zip file like a kind of virtual file, java does not support this idea directly. This idea is not new of far-fetched ( e.g. Windows XP and beyond already supports zip files under this this virtual file concept ).

The Managed File Toolbox allows for total abstraction of the underlying file system and allows for the creation of complex virtual filesystem based upon any other file-similar or file handling technology like FTP, HTTP, Zip and of course the OS native file system.

File vs Managed File

A Managed File is a node in the file system similarly to java.io.File. The main difference is on how to obtain an InputStream or OutputStream from the file. Java IO uses the Adapter Pattern approach to create streams. MiddleHeaven's Managed File uses a simple Factory Method that will return a ManagedFileContent

k A Managed File can be copied to another file or folder simply by invoking copyTo. This can be a boring task in standard Java has no copy utility is provided and we end writing it over and over. Even then, we must choose between the possibility to use the NI/O channels or not. MiddleHeaven tries to use the most efficient copy method according to the real underlying file location and implementation.

The root repositories for files and folders can be obtained by abstracting any supported URI or java.io.File. This task in normally done in bootstrap or configuration code where the application "roots" are configured. We will come back to this when we visit the Bootstrap and Service toolboxes

Model

The figure below illustrates the basic types for the Managed File Toolbox in MiddleHeaven.

Ilustration 1: Managed file main types

The ManagedFile is the main type. From it is possible to obtain other children files, or folders, have access to it parent file or to the ManagedFileContent with in turn provides InputStream and OutputStream to data itself.

Almost all operations throw ManagedIOException, an IOException runtime exception counterpart. Exception handling best practices dictate that all exceptions thrown by operations involving a subsystem or communication with external resources (like files, connections ,etc..) must be checked exception. Here MiddleHeaven makes a trade-off. Normally this exception cannot be resolve only by the application itself, is common to have to have someone reconfigure some port, address, firewall, permission , etc... So there is no much point in checking all exceptions where we do not have means to resolve them. However when we do have means to solve them, or work around them, we can still capture the runtime exception.
MiddleHeaven incorporates the Exception Handler pattern in order to translate common I/O exceptions into more meaningful exception according to best practices (e.g. MiddleHeaven provides FileNotFoundManagedException that receives and holds the file path that was not found in order to be read afterwards for finer exception handling)

All ManagedFile provide the method isWatchable with normally returns false. By contract it only can return true if the ManagedFile implementation also implements WatchableRepository. WatchableRepository allows for watch-dog capabilities to be added by means of registering FileChangeListeners. the watch-dog implementation is provided by the underlying ManagedFile implementation.

Although ManagedFile is the main contract for users of the toolbox, ManagedFileResolver is the most relevant contract for the implementation of the toolbox it self. Because the toolbox must be open to incorporate several different file-like systems and corresponding supporting API MiddleHeaven opts for introducing an abstraction super layer called the engine, represented by the RepositoryEngine interface. Different engines provide different ManagedFileResolvers so any virtual file system could be incorporated.

Use

Using a Managed File is very simple. In this example we create an empty file in the current folder, locate the JUnit.jar in a local maven repository and copy it. As said before, normally the programmer will receive a ManagedFile or ManagedRepository from else where in the system, like a service or via injection. For completion sake we also see in the following example who those "root" files can be obtain from standard java.io.File:

01 02 //ManagedFileRepositories is an integration utilities class that allows 03 // for integration with java.io.File 04 05 ManagedFile folder = ManagedFileRepositories.resolveFile ( new File ( "." )) ; 06 07 assertTrue ( folder.exists ()) ; 08 09 // resolving is simply acquiring a location where the file should be. 10 ManagedFile testJar = folder.resolveFile ( "test.jar" ) ; 11 12 // if exists, delete it. 13 if ( testJar.exists ()){ 14 testJar.delete () ; 15 } 16 17 // create a new (empty) file 18 testJar.createFile () ; 19 20 // it must exist by now 21 assertTrue ( testJar.exists ()) ; 22 23 // manually obtain the path to the repository 24 ManagedFile rep = ManagedFileRepositories.resolveFile ( new File ( System.getProperty ( "user.home" ) + "/.m2/repository/junit/junit/4.1" )) ; 25 26 // resolve the file 27 ManagedFile junitJar = rep.resolveFile ( "junit-4.1.jar" ) ; 28 29 // copy it to the location previous created 30 junitJar.copyTo ( testJar ) ; 31 32 // this a jar file that contains other files. So listFiles() collection will not be empty 33 assertFalse ( junitJar.listFiles () .isEmpty ()) ; 34 35

Code 1: Managed file use example

This example shows how we can abstract any java.io.File into a ManagedFile. It also shows how simple it is to copy a file simply by invoking copyTo. Finally is present a simple example to ilustrate the complete abstraction of the virtual file system of a Zip file by iterating it as we would had done with any other managed file

Under the Hood

The had use the managed file concept in other applications an frameworks I've develop along the year. During this time I've crossed roads with Apache's Commons VFS that thrives on the same "virtual file" idea. Because MiddleHeaven main goal is to abstract implementations and third party libraries I could no use Commons VFS directly. Instead I came up with the RepositoryEngine concept. A repository engine provides a root ManagedFileResolver in order to resolve the location of any file within.

MiddleHeaven Managed File implementation is a almost direct translation of Common VFS witch already provides a vast variety of virtual systems like FTP and ZIP file , to name a few. Plans exist to incorporate WebDav and others.

However Commons VFS does not integrate natively with Commons Upload very frequently used in web applications to mediate file upload logic. This is where MiddleHeaven Managed File Toolbox architecture shines enabling the toolbox to provide the services we need based upon on the code we see most useful, merging or aggregating different third party API when needed. MiddleHeaven Managed File Toolbox provides a Managed File implementation of a ManagedRepository that can be used to interact with the download files in the same exact way we would interact with a file on the disk. We'll talk more about this we we visit the Web Toolbox.