DAO design for writing big XML file on database [on hold]


Kapishin
25/12/2015 9:47 am
HOME ยป Programming Languages

I am currently working on JavaEE application (Spring, Hibernate). I have to put a big XML file (more than 1 gigabyte) on a relational database (Postgres).

The application does not use batch processing. I've done some searching but I did not find any solution for the design of the DAO layer: if I use only one transaction, the server will not response to any request until it finishes the insertion of rows (a huge number of rows). So, using 1 transaction is not a good idea. I can split XML file basing on its tags data: every tag content will be inserted on a row. The idea is to use multithreading to manage transactions (every transaction inserts a defined number of rows). I have found a difficulties to would know how to define the necessary number of transactions to maintain a good time response of the application. I also search how to manage failure of certain transactions. For example, If only 3 transactions write over 1000000 fail, I should try again all the transactions?

When searching, I find that batch processing like Spring batch manages database records and transactions failure. But in my application, we did not use batch processing.

Unfortunately, I can not change the database to Nsql database or add Spring Batch framework to the project.


Related to : DAO design for writing big XML file on database [on hold]
DAO design for writing big XML file on database [on hold]
25/12/2015 9:47 am by Kapishin in Programming Languages

I am currently working on JavaEE application (Spring, Hibernate). I have to put a big XML file (more than 1 gigabyte) on a relational database (Postgres).

The application does not use batch processing. I've done some searching but I did not find any solution for the design of the DAO laye


why we need to check for errors while writing a file in C? [on hold]
7/1/2015 1:06 pm by rhinojosa in Programming Languages

what is the need of checking for errors in this program in line 7.

 #include <stdio.h>
    int main()
    {
       int n;
       FILE *fptr;
       fptr=fopen("C:\program.txt","w");
       if(fptr==NULL){
          printf("Error!");   
          exit(1);             
       
writing data to a file in runtime [on hold]
1/12/2014 10:06 am by Frank Bradley in Programming Languages

i found the more specific error: actually i want to write to a file while running the script but it is so happening that the content is written to the file only when the script ends. I am taking data from a sensor and writing to a file and at the same time reading data from the file to process TAGS: writing data file runtime hold

Database design to hold multiple iteration measurements
by Thomas Gueze in Programming Languages

I am new to sqlite and SQL in general. I am keen to switch from flat-files to sqlite for holding some measurement information. I need a tip on how to better layout the database, since I have zero experience with this.


I have a ~10000 unique statistic counters that are collected before


Database design to hold a person's information that changes with time?
by ChrisMe in Programming Languages

We use a third-party product to manage our sports centre membership. We have several membership types (eg. junior, student, staff, community) and several membership statuses (eg. annual, active, inactive, suspended). Unfortunately the product only records a member's current membership type and st


Mysql database design concept for a hospital [on hold]
30/12/2014 7:01 pm by Progdis in Programming Languages

I am developing a hospital Management Application in mysql and php. I am giving brief of tables for brevity. I have among other tables for

medicine

id, medicine_name, medicine_id

patient_table

id, general_regn_no, ipd_regn_no, patient_nam
Database Design for book ,author and editor [on hold]
19/1/2015 2:20 am by Miklebud in Programming Languages

1)Author public class Author {

Integer id;
String  name;

} 2)Editor public class Editor {

Integer id;
String  name;

} 3)Book import java.util.List;

public class Book {

Integer      id;

String       title;
Design database to hold html, jsp, java, javascript, etc code
by Vytautas in Programming Languages

i want to build a small tool to save useful code snippet in the database. But don't know which kind of datatype should I use to hold all the codes. Is there problem with special characters? can I use String to hold code? can you please give me some advice? thank you in a


What is the acceptable level of Granularity for Application database design? [on hold]
4/12/2014 7:05 pm by hyperNURb in Programming Languages

In my opinion there should be a discipline that seeks to throw more light on the Psychological aspect of Information And Communication Technology as a career, IT Psychology maybe. The Reason for the above comment is because in a recent forum discussion relating to database design, a fellow forum


How to give a client backups of their data without compromising our database design? [on hold]
by Javed in Programming Languages
The client is asking for backups of their data, however we would not like to share our database design as it may lead to a security risk...inb4: Security by Obscurity is not a valid security control method.I have thought about dumping only pertinent tables with their data in csv or excel xml form. W
Privacy Policy - Copyrights Notice - Feedback - Report Violation - RSS 2014 © bighow.org All Rights Reserved .