Ffpojo

FFPOJO - Flat File POJO Parser ##

The FFPOJO Project is a Flat-File Parser, POJO based, library for Java applications. It's a Object-Oriented approach to work to flat-files, because the libray is based on POJOs and an Object-Flat-Mapping (OFM), using Java Annotations, XML or both. When used together, the XML mapping overrides the annotations.

The FFPOJO library can work to positional and delimited flat-files, and provides many features like:

• Annotations and/or XML mappings, with overridding (like the JPA api)
• Uses a "Metadata Container" pattern, that caches metadata information in static way to improve performance on parsing
• Simple text-to-pojo and pojo-to-text parsing
• Easy custom-type conversion when reading or writing, through the decorator API
• Full object-oriented flat-file reader that supports complex files with Header, Body and Trailer definitions
• File system flat-file reader developed with NIO, that performs 25% faster than regular buffered reader
• Record processor layer, that works in "Push" model and supports single-threaded or multi-threaded file processing
• Full object-oriented flat-file writer
• Lightweight, no dependence to any other framework or API

Starting Points

• Releases
• Getting Started
• Samples
• XML Schema

---

Getting Started Tutorial <a id="getting-started"/>

This is a samples based tutorial, that shows the most important features of FFPOJO in a hurry. To run the sample codes you will need to download the sample text file resources, that are used in the codes. Download full samples in project named ffpojo-samples.

Building The Project

When comming to GitHub this project was migrated to Maven, then building the project is very simple, default maven style. This project also have no dependencies, so thats simplier at most. The build process is shown below:

1. Install Apache Maven (version 2+)
2. Append the Maven bin path to your PATH environment variable
3. Open the command line and go to the ffpojo folder that have the "pom.xml" file
4. Enter the command below: mvn clean install
5. Now its just pick-up the generated jar file at the "target" folder.

---

Metadata Definitions

To describe your flat-file records and fields you must use Java Annotations and/or XML. If used both, the XML information will override the annotations on a class level. It means that you can't override a single field, but you can override an entire record class.

---

Record Class

The record class is a Plain-Old-Java-Object that represents a line in a flat-file. One record can be Positional or Delimited (not both). The only rule is that the record class must be in JavaBeans standard, providing one getter (pojo-to-text parsing) and one setter (text-to-pojo parsing) for each field.

---

Record Field

The record field is record class attribute that represents a field-token in a flat-file. Positional records has Positional Fields and Delimited Records has Delimited Fields (we can't mix cats and dogs). By convention, the annotation metadata for fields must appear in the GETTER METHODS.

---

Object-Flat-Mapping XML Schema

If you choose to map your ffpojos using XML, it must be valid according to the ffpojo-ofm.xsd schema.

---

Object-Flat-Mapping XML Example

<ffpojo-mappings xmlns="http://www.ffpojo.org/ofm">
	<ffpojo class="com.domain.pkg.Customer">
		<positional>
			<positional-field name="name" initial-position="1" final-position="5"/>
			<positional-field name="address" initial-position="6" final-position="10"/>
		</positional>
	</ffpojo>
	<ffpojo class="com.domain.pkg.Employee">
		<delimited delimiter=";">
			<delimited-field name="id" position-index="1" decorator-class="com.domain.pkg.MyLongDecorator"/>
			<delimited-field name="endereco" position-index="2"/>
		</delimited>
	</ffpojo>
</ffpojo-mappings>

---

Index of Samples <a id="samples"/>

1. Simple Positional Record Parsing Example
2. Simple Delimited Record Parsing Example
3. Positional Record Parsing With Decorator Example
4. Simple File System Flat File Reader Example
5. Simple Input Stream Flat File Reader Example
6. File System Flat File Reader With Header And Trailer Example
7. Simple File System Flat File Reader With Default Flat File Processor Example
8. Simple File System Flat File Reader With Thread Pool Flat File Processor Example
9. Simple File System Flat File Reader With Thread Pool Flat File Processor And Error Handler Example
10. Simple File System Flat File Writer Example

---

1 - Simple Positional Record Parsing Example <a id="example-1"/>

// package and imports omitted

public class SimplePositionalRecordParsingExample {

	private static final String INPUT_TXT_RESOURCE_CLASSPATH = "SimplePositionalRecordParsingExampleInput.txt";

	//change here (make sure you have permission to write in the specified path):
	private static final String OUTPUT_TXT_OS_PATH = "C:/Users/gibaholms/Desktop/SimplePositionalRecordParsingExampleOutput.txt";

	@PositionalRecord
	public static class Customer {

		private Long id;
		private String name;
		private String email;

		@PositionalField(initialPosition = 1, finalPosition = 5)
		public Long getId() {
			return id;
		}
		public void setId(Long id) {
			this.id = id;
		}
		// must use a String setter or a FieldDecorator
		public void setId(String id) {
			this.id = Long.valueOf(id);
		}

		@PositionalField(initialPosition = 6, finalPosition = 25)
		public String getName() {
			return name;
		}
		public void setName(String name) {
			this.name = name;
		}

		@PositionalField(initialPosition = 26, finalPosition = 55)
		public String getEmail() {
			return email;
		}
		public void setEmail(String email) {
			this.email = email;
		}
	}

	public static void main(String[] args) {
		SimplePositionalRecordParsingExample example = new SimplePositionalRecordParsingExample();
		try {
			System.out.println("Making POJO from TXT...");
			example.readCustomersFromText();

			System.out.println("Making TXT from POJO...");
			example.writeCustomersToText();

			System.out.println("END !");
		} catch (IOException e) {
			e.printStackTrace();
		} catch (FFPojoException e) {
			e.printStackTrace();
		}
	}

	public void readCustomersFromText() throws IOException, FFPojoException {
		FFPojoHelper ffpojo = FFPojoHelper.getInstance();
		BufferedReader textFileReader = new BufferedReader(new InputStreamReader(getClass().getClassLoader().getResourceAsStream(INPUT_TXT_RESOURCE_CLASSPATH)));
		String line;
		while ( (line = textFileReader.readLine()) != null) {
			Customer cust = ffpojo.createFromText(Customer.class, line);
			System.out.printf("[%d][%s][%s]\n", cust.getId(), cust.getName(), cust.getEmail());
		}
		textFileReader.close();
	}

	public void writeCustomersToText() throws IOException, FFPojoException {
		FFPojoHelper ffpojo = FFPojoHelper.getInstance();
		File file = new File(OUTPUT_TXT_OS_PATH);
		file.createNewFile();
		BufferedWriter textFileWriter = new BufferedWriter(new FileWriter(file));
		List<Customer> customers = createCustomersMockList();
		for (int i = 0; i < customers.size(); i++) {
			String line = ffpojo.parseToText(customers.get(i));
			textFileWriter.write(line);
			if (i < customers.size() - 1) {
				textFileWriter.newLine();
			}
		}
		textFileWriter.close();
	}

	private static List<Customer> createCustomersMockList() {
		List<Customer> customers = new ArrayList<Customer>();
		{
			Customer cust = new Customer();
			cust.setId(98456L);
			cust.setName("Axel Rose");
			cust.setEmail("axl@thehost.com");
			customers.add(cust);
		}
		{
			Customer cust = new Customer();
			cust.setId(65478L);
			cust.setName("Bono Vox");
			cust.setEmail("bono@thehost.com");
			customers.add(cust);
		}
		{
			Customer cust = new Customer();
			cust.setId(78425L);
			cust.setName("Bob Marley");
			cust.setEmail("marley@thehost.com");
			customers.add(cust);
		}
		return customers;
	}

}

---

2 - Simple Delimited Record Parsing Example <a id="example-2"/>

// package and imports omitted

public class SimpleDelimitedRecordParsingExample {

	private static final String INPUT_TXT_RESOURCE_CLASSPATH = "SimpleDelimitedRecordParsingExampleInput.txt";

	//change here (make sure you have permission to write in the specified path):
	private static final String OUTPUT_TXT_OS_PATH = "C:/Users/gibaholms/Desktop/SimpleDelimitedRecordParsingExampleOutput.txt";

	@DelimitedRecord(delimiter = "|")
	public static class Customer {

		private Long id;
		private String name;
		private String email;

		@DelimitedField(positionIndex = 1)
		public Long getId() {
			return id;
		}
		public void setId(Long id) {
			this.id = id;
		}
		// must use a String setter or a FieldDecorator
		public void setId(String id) {
			this.id = Long.valueOf(id);
		}

		@DelimitedField(positionIndex = 2)
		public String getName() {
			return name;
		}
		public void setName(String name) {
			this.name = name;
		}

		@DelimitedField(positionIndex = 3)
		public String getEmail() {
			return email;
		}
		public void setEmail(String email) {
			this.email = email;
		}
	}

	public static void main(String[] args) {
		SimpleDelimitedRecordParsingExample example = new SimpleDelimitedRecordParsingExample();
		try {
			System.out.println("Making POJO from TXT...");
			example.readCustomersFromText();

			System.out.println("Making TXT from POJO...");
			example.writeCustomersToText();

			System.out.print