I'm using in my application a mix Spring/Hibernate (nothing original). For a given feature, I have to import the content of a CSV file into a table of my Oracle DB. For now, I juste create the objects, I do
on each of them (I need to retrieve their newly allocated Id)
Then the transaction occurs at the end of the method, using the Spring transaction API.
Everything works fine, except performance, which is correct for some 5000's objects, but not for 100 000...
So I look for ideas to accelerate this stuff. I've heard of bulk inserts with Hibernate, but could not find any solid reference. Can anybody give me some ideas to perform this import with greater performance?
Spring / Hibernate / JUnit - No Hibernate Session bound to Thread
read file in classpath
Intent of Spring form command
Can a Spring form command be a Map?
every 100 or 1000 inserts..
Spring: No transaction manager has been configured
That will flush and clear the hibernate session and stop it growing too big (possibly why your 100 000 objects are taking so long)..
Acquiring a spring configuration value from a custom class
Furthermore if you're using identity identifier generator hibernate will silently turn batch inserts off.
How do you make a multi-module spring configuration?
Batch inserts will improve performance.
How to XML configure Spring bean for constructor injection when bean has varargs constructor
You'd also need to specify the hibernate.jdbc.batch_size configuration property equivalent to your 100 at a time number.. Manning's Java Persistence with Hibernate was the source of this (great book - saved my skin numerous times)..
StatelessSession ss=sessionFactory().openStatelessSession(); Transaction tx=ss.beginTransaction();
Especially batch operations are often more performantly executed using plain old JDBC.
This of course depends on a variety of conditions but you should at least see this as an option and compare performance of both approaches..