JEX

Description

Java-based software and GUI to automatically manage/database large numbers of files (e.g., images) in a transparent file structure and perform batch processing with standard tools (ImageJ, R, Octave, Weka).

Getting Java...

Google "Download Java 1.8 Oracle", following the link for the Java (v1.8.X) SE standard development kit. Accept the license, download and install.

Downloading the JEX Executable Version

Executable jars and simple exectuable scripts to run those jars with specific java heap space arguments for increasing the amount of RAM that can be used are available for download at sourceforge. Simply save and unzip the file anywhere you would like (except maybe your root directory of your hard drive such as 'C:/' as that is often weirdly write protected and can cause issues) and double-click the "JEX for Mac" or "JEX for Windows" exectubale script depending on your operating system (Linux users can double-click the jar directly or run the jar from the command line to change the memory settings). This will start a terminal window that will show information regarding any actions that JEX is taking as well as error information that would be very useful for debugging issues you might run into :-).

You can update your version of JEX to the most recent stable release or developmental version at any time by simply clicking the "Update" button in the top-left corner of JEX when viewing a database. However, to create and view a database, follow the mini-tutorial below titled "Getting Started".

Reporting Bugs/Issues

Please go to https://github.com/jaywarrick/JEX/issues to describe and submit the issue, which will be emailed to JEX's developers.

Getting started

Note: There is also a "Test Script" with screen shots created for a specific project, that you can also download and follow here.

Upon starting JEX, you will be shown a screen for either...

1. Creating a user profile
2. Opening a user profile via a file browser

or

3. Opening a recently opened user profile

After creating your user profile, typically you just click the recently opened profile to use JEX. However, upon first use, you must create a user profile. Select to create a user profile and choose to save the user profile wherever you like (e.g., your "Documents" folder).

With the user profile file created, we can now store information regarding what disks or servers you would like to use for storing JEX databases (e.g., any folder on a NAS device, external hardrive, or your PC). These locations are remembered so you don't have to constantly relocate them each time you open JEX. These locations are termed JEX "Repositories". This is the highest level of organization of data.

• A small side note: The organization of data by JEX goes like this... Repositories > Databases > Datasets > Entries > Objects. Databases are simply folders of Datasets with some extra xml files that represent the database structure. Datasets are simply folders Entries. Entries are simply folders of Objects. Objects are simply folders of raw data files (e.g., tiff image files) and a single small "arff" format text file that JEX uses to keep track of how to organize the raw data files in that folder (e.g., many tiff files that represent timpoints from a timelapse). In this way, the files contained within an Object folder can be treated as a N-dimensional dataset. For example, a group of tiff files could be images that represent a mosaic of images taken in multiple colors, at multiple z-planes, at multiple times. The small .arff file keeps track of which images are for which location in the mosaic as well as which color, plane, and time. The number and name of dimensions of any Object is completely user defined in JEX.

With a respository location defined for your user profile, we can now create databases using the "+" button. Name the database. Single-click the icon to the left of the database name to open the newly created and empty database. Once you have opened your database the general rule of thumb is to procede through the "tabs" of JEX in order (see the icons numbered 1-7 in the top center of the window).

You are initially brought into JEX with tab #1 active. At the bottom center of the window is a button labeled "Add Dataset". As the name implies, you can click this to create a Dataset.

• Another side note: A Dataset is a 2D array of Entries and is meant to represent something like a well-plate commonly used for cell culture (e.g., a 96-well plate consisting of 8 rows and 12 columns from which one could collect similar types of imaging data); however, feel free to organize your data as you see fit. Each Entry (or well of the well plate) generally will have the same types of objects (e.g., an N-dimensional image object of the image data obtained from that well and a resulting table of data resulting from it's analysis) but each object is unique to that Entry or well. Given each Entry contains data that is independent of data from adjacent Entries but data Objects within each Entry typically depend on one-another. Thus, multi-threading of data analysis is performed at the level of Entries (i.e., analysis of Entries is processed in parallel using the multiple processing cores of the host computer while operations within each Entry are processed linearly/sequentially).

Long story short, you create a Dataset, which is a 2D array of Entries. Now we can import data into each Entry. You can see your array of entries by going to tab #2. Each Entry is displayed as a box. Click the boxes to select particular Entries you would like to import data into (e.g., all of them or just particular row, column, or individual Entry). You can also use the "Quick Selector" on the left side to select locations in the Dataset array.

To import data go to tab #3. On the right-hand side of this tab there are selections to be made. Generally work from top to bottom. Choose what type of object you will be creating (e.g., and Image object). Then select the number of dimensions (e.g., 1D for a time-series of images or 2D for a timeseries of images taken in different colors, etc.). Type a name to give to this object. Choose the files that make up this object.

• Side note: Within the "Choose files" dialog, click the "load" button, choose the files you would like using the file browser and click "OK".

Then below the list that shows the files you have chosen, indicate how the files are ordered in the list. Using the example of a 2D image object that has 12 image files from 3 different times in 4 different colors, if the first 3 files are images from different times of the same color while images 4-7 are from those same times but in the second color, you would make the following selections... Type a name for the "Time" dimension into the first drop-down (e.g., "T" or "time" or "Time", etc. it is your choice) and enter 3 for the "size" of the dimension. Then type a name for the "Color" dimension and a "size" of 4. Essentially you are indicating how to assign dimension values to each file in the list (the first dimension listed in the inner-most loop of a nested for-loop of dimensions used for assigning dimension indices and so on).

You must also indicate when the file list changes to a different "Array Row" and "Array Col", if at all. "Array Row" and "Array Col" are reserved dimension names for the Dataset rows and columns. If you are importing into a single Entry, it doesn't matter where in the list "Array Row" and "Array Col" fall in the list as there is only one Entry into which data is being imported so the index never changes. However, if you have multiple Entries, pay attention to where in the list these special dimension names fall. Although not completely obvious, it doesn't matter what you put for the "size" of the "Array Row" and "Array Col" dimensions as this is inferred based on your selection to reduce the potential for errors.

When you feel you have the correct selections made, click "Deal files". This displays within the Entry boxes themselves, which files will be put where and what dimension values they will be assigned. If this doesn't look right, review your selections and adjust accordingly. When the result of "Deal files" looks right, click the "Save" or "Import" button at the bottom of the window to actually import these files to create an N-Dimensional object.

The newly imported object with the name you specified will now show up in the left-hand portion of the window that is everpresent in all tabs of JEX. If this is an image object and you have an Entry selected into which you created such an object, you can double-click the image icon to open the N-dimensional image in JEX's built-in image viewer.

Other potential ways to specify files for importing is to drag files onto Entry boxes when using tab #3 or by dragging files into the list above the "Choose files" button of the same tab. Whatever is listed in the Entry boxes is how things will be imported when the "Save" or "Import" button at the bottom of the window is clicked.

• Side Note: In general, save the database frequently, in between steps, by clicking the "Save" button in the top-left of the JEX window.

A useful but not a required step would then be to use tab #4 to create "Label" objects in entries to save information regarding experimental conditions. For example, you can create a label object with the name "Treatment Concentration" in each Entry. The value of the label object would then be the actual treatment concentration used to obtain the data for that Entry. These label objects can then be passed to batch processing algorithms that might benefit from such information.

Once you have an object in the d