python - Merge Columns and Remove Duplication -

python - Merge Columns and Remove Duplication -

i have input file has info in 2 columns. need merge both columns , remove duplication. suggestions how start ? !

input file

5045 2317 5045 1670 5045 2156 5045 1509 5045 3833 5045 1013 5045 3491 5045 32 5045 1482 5045 2495 5045 4280 5045 1380 5045 3998

expected output

5045 2317 1670 2156 1509 3833 1013 3491 32 1482 2495 4280 1380 3998

to maintain order:

from itertools import chain open("in.txt") f: lines = list(chain.from_iterable(x.split() x in f)) open("in.txt","w") f1: ind, line in enumerate(lines,1): if not line in lines[:ind-1]: f1.write(line+"\n")

output:

5045 2317 1670 2156 1509 3833 1013 3491 32 1482 2495 4280 1380 3998

if order not matter:

from itertools import chain open("in.txt") f: lines = set(chain.from_iterable(x.split() x in f)) open("in.txt","w") f1: f1.writelines("\n".join(lines))

if there 1 number repeated in first column:

with open("in.txt") f: col_1 = f.next().split()[0] # first column number lines = set(x.split()[1] x in f) # sec column nums lines.add(col_1) # add together first column num open("in.txt","w") f1: f1.writelines("\n".join(lines))

python file merge duplication