SkillAgentSearch skills...

TestFileAppendSpeed

java文件写入

GenerateConvertImprove

Install / Use

/learn @jackuhan/TestFileAppendSpeed
About this skill

Quality Score

0/100

Supported Platforms

Universal

README

一、如何写入文件到硬盘I/O

首先了解下换行符在不同平台不一样 String line = System.lineSeparator(); if ("\r\n".equals(line)) { System.out.println("windows"); } else if ("\n".equals(line)) { System.out.println("Mac"); }else  if ("\r".equals(line)) { System.out.println("linux/unix"); }

1、FileOutputStream

public static void method2(String file, String conent) {
BufferedWriter out = null;
try {
out = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream(file, true)));
out.write(conent+"\r\n");
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}

2、RandomAccessFile + NIO内存映射文件

public static void method3(String fileName, String content) {
try {
// 打开一个随机访问文件流，按读写方式
RandomAccessFile randomFile = new RandomAccessFile(fileName, "rw");
// 文件长度，字节数
long fileLength = randomFile.length();
// 将写文件指针移到文件尾。
randomFile.seek(fileLength);
randomFile.writeBytes(content+"\r\n");//不同平台的换行符不一样
randomFile.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}

https://blog.csdn.net/jdsjlzx/article/details/51648044

RandomAccessFile的绝大多数功能，但不是全部，已经被JDK 1.4的NIO的”内存映射文件(memory-mapped files)”给取代了，你该考虑一下是不是用”内存映射文件”来代替RandomAccessFile了。

内存映射文件(memory-mapped file)能让你创建和修改那些大到无法读入内存的文件。

public class LargeMappedFiles { static int length = 0x8000000; // 128 Mb

public static void main(String[] args) throws Exception {
    *// 为了以可读可写的方式打开文件，这里使用RandomAccessFile来创建文件。*
    FileChannel fc = new RandomAccessFile("test.dat", "rw").getChannel();
    *//注意，文件通道的可读可写要建立在文件流本身可读写的基础之上*
    MappedByteBuffer out = fc.map(FileChannel.MapMode.READ_WRITE, 0, length);
    *//写128M的内容*
    for (int i = 0; i < length; i++) {
        out.put((byte) 'x');
    }
    System.out.println("Finished writing");
    *//读取文件中间6个字节内容*
    for (int i = length / 2; i < length / 2 + 6; i++) {
        System.out.print((char) out.get(i));
    }
    fc.close();
}

}

3、FileWriter BufferedWriter PrintWriter

1、FileWriter

https://sunnylocus.iteye.com/blog/694666?page=2

  /** 
     * 将信息记录到日志文件 
     * @param logFile 日志文件 
     * @param mesInfo 信息 
     * @throws IOException  
     */  
public void logMsg(File logFile,String mesInfo) throws IOException{  
if(logFile == null) {  
  throw new IllegalStateException("logFile can not be null!");  
  }  
 Writer txtWriter = new FileWriter(logFile,true);  
 txtWriter.write(dateFormat.format(new Date()) +"\t"+mesInfo+"\n");  
 txtWriter.flush();  
}  

2、BufferedWriter

https://github.com/itgoyo/LogToFile/blob/master/logtofilelibrary/src/main/java/com/itgoyo/logtofilelibrary/LogToFileUtils.java

try {
BufferedWriter bw = new BufferedWriter(new FileWriter(logFile, true));
bw.write(logStr);
bw.write("\r\n");
bw.flush();
} catch (Exception e) {
Log.e(tag, "Write failure !!! " + e.toString());
}

https://blog.csdn.net/high2011/article/details/49620125  FileWriter fw=new FileWriter("d:/x.log",true);//true代表不覆盖文件的内容，而是紧跟着添加内容 BufferedWriter bw=new BufferedWriter(fw); bw.writer(message); //message是String类型的参数 bw.close() ; //关闭流 fw.close();   //关闭流 区别：如果同时使用bw fw，那么性能会大大提高，而单独使用FileWriter操作字符，每写一次数据，磁盘就有一个写操作，性能很差。 如果加了缓冲，那么会等到缓冲满了以后才会有写操作，效率和性能都有很大提高。

3、PrintWriter

https://blog.csdn.net/flamezyg/article/details/5190796  PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter("foo.out"))); PrintWriter pw = new PrintWriter(System.out); PrintWriter out = new PrintWriter(new BufferedOutputStream(new FileOutputStream("foo.out")));  将缓冲 PrintWriter 对文件的输出。如果没有缓冲，则每次调用 print() 方法会导致将字符转换为字节，然后立即写入到文件，而这是极其低效的。

https://blog.csdn.net/qq1175421841/article/details/52411607  Socket编程中,尽量用PrintWriter取代BufferedWriter

  1. PrintWriter的print、println方法可以接受任意类型的参数（字符，字节流），而BufferedWriter的write方法只能接受字符、字符数组和字符串；

  2. PrintWriter的println方法自动添加换行，BufferedWriter需要显示调用newLine方法；

  3. PrintWriter的方法不会抛异常，若关心异常，需要调用checkError方法看是否有异常发生；

  4. PrintWriter构造方法可指定参数，实现自动刷新缓存（autoflush）；

  5. PrintWriter的构造方法更广。

4、flush

不使用flush() String s = "Hello World"; try { PrintWriter pw = new PrintWriter(System.out); pw.write(s); // pw.flush(); } catch (Exception ex) { ex.printStackTrace(); }

输出： buffer没有满，输出为空。 使用flush() String s = "Hello World"; try { PrintWriter pw = new PrintWriter(System.out); pw.write(s); pw.flush(); } catch (Exception ex) { ex.printStackTrace(); } 得到期望的输出结果。

5、close()和flush()作用有交集！

public static void main(String[] args) { BufferedWriter fw =null; try { fw = new BufferedWriter(new FileWriter("e:\test.txt")); fw.write("wo shi lucky girl."); //fw.flush(); fw.close(); } catch (Exception e) { e.printStackTrace(); } } //fw.flush();这句有和无并不影响输出结果，不太明白词句是否必要？ 因为close的时候，会把你没flush掉的一起flush掉。 缓冲区中的数据保存直到缓冲区满后才写出，也可以使用flush方法将缓冲区中的数据强制写出或使用close()方法关闭流，关闭流之前，缓冲输出流将缓冲区数据一次性写出。在这个例子中，flash()和close()都使数据强制写出，所以两种结果是一样的，如果都不写的话，会发现不能成功写出

6、Java默认缓冲区大小是多少？

默认缓冲去大小8192字节。

4、Java 各种文件writer 性能大比拼

http://ju.outofmemory.cn/entry/181638 Map File’s performance is the highest

  • RandomAccessFile with ByteBuffer:153600 test2.tmp spent time=508ms 

  • RandomAccessFile with ByteBuffer:1048576 test3.tmp spent time=422ms 

  • RandomAccessFile + with MappedByteBuffer and FileMap:104857600 test4.tmp spent time=110ms 

  • DataOutputStream with BufferedOutputStream :153600 test5.tmp spent time=1103 ms 

  • DataOutputStream without BufferedOutputStream :153600 test6.tmp spent time=680543 ms 

  • PrintWriter/FileWriter with BufferedWriter:153600 test7.tmp spent time=2209 ms 

  • PrintWriter/FileWriter without BufferedWriter:153600 test8.tmp spent time=2284 ms 

  • PrintWriter/OutputStreamWriter with BufferedWriter:153600 test9.tmp spent time=1326 ms 

  • PrintWriter/OutputStreamWriter without BufferedWriter:153600 test10.tmp spent time=2336 ms

最快代码

test3("test4.tmp", 1000000, 1024 * 1024 * 100);

RandomAccessFile raf1 = new RandomAccessFile(file, "rw");     FileChannel fc = raf1.getChannel();     MappedByteBuffer raf = fc.map(MapMode.READ_WRITE, 0, mapsize);     raf.clear();     // ByteBuffer raf = ByteBuffer.allocateDirect(mapsize);     byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };     byte[] utfstr = "this is a test".getBytes("UTF-8");     long s = System.nanoTime();     long count = 0;     for (int i = 0; i < loop; i++) {       if (raf.remaining() < 140) {         System.out.println("remap");         count += raf.position();         raf = fc.map(MapMode.READ_WRITE, count, mapsize);         // raf = fc.map(MapMode.READ_WRITE, raf.position(),         // raf.position()+mapsize);       }       raf.put(b1);       raf.putInt(i);       raf.putInt(i + 1);       raf.put(utfstr);       raf.put((byte) '\n');     }     fc.close();     raf1.close();

全部代码

public class TestAppendSpeed {
  public static void main(String[] args) throws Exception {
    // Raw RandomAccessFile
    // it is too slow to comment it
    // test1("test1.tmp",100000);
    // nio ByteBuffer RandomAccessFile
    test2("test2.tmp", 1000000, 1024 * 150);
    // nio ByteBuffer RandomAccessFile, with big buffer
    test2("test3.tmp", 1000000, 1024 * 1024);
    // using map file with MappedByteBuffer
    test3("test4.tmp", 1000000, 1024 * 1024 * 100);
    // old io, BufferedOutputStream with BufferedOutputStream
    test4("test5.tmp", 1000000, 1024 * 150);
    // old io, BufferedOutputStream without BufferedOutputStream
    // it is too slow to comment it
    // test5("test6.tmp", 1000000, 1024 * 150);
    // old io, PrintWriter/FileWriter with BufferedWriter
    test6("test7.tmp", 1000000, 1024 * 150);
    // old io, PrintWriter/FileWriter without BufferedWriter
    test7("test8.tmp", 1000000, 1024 * 150);
    // old io, PrintWriter/OutputStreamWriter without BufferedWriter
    test8("test9.tmp", 1000000, 1024 * 150);
    // old io, PrintWriter/FileWriter without BufferedWriter
    test9("test10.tmp", 1000000, 1024 * 150);
  }
  public static void test9(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    PrintWriter raf = new PrintWriter(new OutputStreamWriter(
        new FileOutputStream(file, true)));
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.print(b1);
      raf.print(i);
      raf.print(i + 1);
      raf.print(utfstr);
      raf.print((byte) '\n');
    }
    raf.close();
    long d = System.nanoTime() - s;
    System.out
        .println("PrintWriter/OutputStreamWriter without BufferedWriter:"
            + mapsize
            + " "
            + file
            + " spent time="
            + (d / 1000000)
            + " ms");
  }
  public static void test8(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    PrintWriter raf = new PrintWriter(new BufferedWriter(
        new OutputStreamWriter(new FileOutputStream(file, true)),
        mapsize));
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.print(b1);
      raf.print(i);
      raf.print(i + 1);
      raf.print(utfstr);
      

jackuhan

View profile

View on GitHub
GitHub Stars7
CategoryDevelopment
Updated3y ago
Forks0
jackuhan/TestFileAppendSpeed

Languages

Java

Security Score

50/100

Audited on Mar 5, 2023

No findings