Monday, March 26, 2012

duplicate in text file

I have my SQL server reads a text file into a temp table, then it copies
that data over to a main table, but it looks to see if a particular filed
has a certain value, if it does, it updates that record instead of creating
a duplicate.
But, what I am finding is that if the text file has duplicate records in it,
it will duplicate into the main table.
Any ideas on how I can weed out duplicates from the text file that I am
importing?You can add an intermediate process to remove the duplicates after they are
loaded into the temp table and before perorming the insert or the update to
the main table.
"Johnfli" wrote:

> I have my SQL server reads a text file into a temp table, then it copies
> that data over to a main table, but it looks to see if a particular filed
> has a certain value, if it does, it updates that record instead of creatin
g
> a duplicate.
> But, what I am finding is that if the text file has duplicate records in i
t,
> it will duplicate into the main table.
> Any ideas on how I can weed out duplicates from the text file that I am
> importing?
>
>|||Can't you insert everything into the temp table -- the do the SELECT
DISTINCT at the temp table? prior to entering into the main table
Jack Vamvas
________________________________________
__________________________
Receive free SQL tips - register at www.ciquery.com/sqlserver.htm
SQL Server Performance Audit - check www.ciquery.com/sqlserver_audit.htm
New article by Jack Vamvas - SQL and Markov Chains -
www.ciquery.com/articles/art_04.asp
"Johnfli" <john@.ivhs.us> wrote in message
news:%23SPhtYdLGHA.2216@.TK2MSFTNGP09.phx.gbl...
> I have my SQL server reads a text file into a temp table, then it copies
> that data over to a main table, but it looks to see if a particular filed
> has a certain value, if it does, it updates that record instead of
creating
> a duplicate.
> But, what I am finding is that if the text file has duplicate records in
it,
> it will duplicate into the main table.
> Any ideas on how I can weed out duplicates from the text file that I am
> importing?
>

No comments:

Post a Comment