Breadcrumbs

[comathi]

Here's what I came up with... it works with 10 lines of text in each file... now let's see if it can handle 10 million :lol:


This file is hosted off-site.

[clanc789]

It works! I already cleared up not needed lines so the 14 million decreased to 11 million. My 8 million was already cleared up so now I have 19 million in 1 list I'm so happy

Since I had a lot of lists to add, there might/are some double/triple/etc entries. Anyone knows how to search for them and replace them with a blank line/just delete that whole line, besides manually?

[comathi]

Here you go. I cooked up another quick little application for you... I hope you like the names so far :lol:


This file is hosted off-site.

[clanc789]

Running it now, it'll be checking 19 million lines and I am timing it, so I'll give you the results ASAP

A big and huge thanks already!

Also, notepad++ was working ~4 hours to put the list in alphabetical order and then just crashed. Anyone got a piece of code to do that/link so I can make a program for it?

[comathi]

I'll see what I can do about that after lunch. It'll be a bit more complicated than the other two, but this is where my classes on recursive algorithms will come in handy

[clanc789]

'lunch' is weird to read lol, its 17:59 here (Holland) xD Have a good meal

[comathi]

Well it was 11:59 here in eastern Canada lol

Anyway, turns out VB.NET had built in functions for sorting arrays, so this took way less time than I thought it would.

Introducing the newest member of Comathi's custom line of dictionary tools :lol:


This file is hosted off-site.

[clanc789]

My PC restarted due to updates or something, will run both programs tomorrow! Thanks already!

[clanc789]

The program is running 14 hours now, and no sign of the 'complete' message. Lets hope it comes soon xD (I'm talking about Unify). I took a wild guess and thought that every line would be scanned to search for a duplicate version, resulting in 19.000.000^2 = 361.000.000.000.000 'checks'. Estimating that my PC does ~20.000 entries each second that would take 204 days to complete.

So I guess that I'll leave the double versions within the file xD

[comathi]

The processing time required to check one element gets bigger exponentially as more items are added to the array it needs to check every time.

I don't think it'll be 204 days, but expect it to take quite some time ;)