Remove duplicated lines in editor

You have encountered a problem using Far Manager? Here you can get help.
Post Reply
KlepetoX
Posts: 129
Joined: Sun 08 Jan, 2012 08:23
Location: Czech Republic
Has thanked: 2 times
Been thanked: 1 time

Remove duplicated lines in editor

Post by KlepetoX »

Hello, I need to remove duplicated lines in editor without resorting and leaving last duplicated row. I found a regular expression for notepad++, which works. But I have to modify expression for FAR and I don't know how.
The regular expression for search is: ^(.*?)$\s+?^(?=.*^\1$)
Replace expression is empty.
Thanks
Shmuel
Posts: 6836
Joined: Thu 23 Mar, 2006 21:36
Location: Israel
Has thanked: 41 times
Been thanked: 530 times

Remove duplicated lines in editor

Post by Shmuel »

You can try out my Duplicate Fighter (search for it in the forum).
User avatar
HaRT
Moderator
Posts: 10863
Joined: Tue 30 Aug, 2005 17:21
Has thanked: 221 times
Been thanked: 358 times

Remove duplicated lines in editor

Post by HaRT »

KlepetoX wrote: Tue 11 Oct, 2022 10:31 The regular expression for search is: ^(.*?)$\s+?^(?=.*^\1$)
AFAIK, FAR doesn't support multi-line regular expressions. Yet e.g. the RESearch plugin does, you can find its announcements in the Russian-language part of the forum.
Фар есть инструмент, а не нянька. © 2009 DrKnS
KlepetoX
Posts: 129
Joined: Sun 08 Jan, 2012 08:23
Location: Czech Republic
Has thanked: 2 times
Been thanked: 1 time

Remove duplicated lines in editor

Post by KlepetoX »

Shmuel wrote: Tue 11 Oct, 2022 10:49 You can try out my Duplicate Fighter (search for it in the forum).
It seems to work well, but I need to keep last duplicated line and delete all before. See at example:

line 1
line 2
line 3
line x
line 3
line 1


should be:

line 2
line x
line 3
line 1


not:

line 1
line 2
line 3
line x


It should read file from the end to beginning instead of from beginning to the end. Maybe as an option in settings.
Shmuel
Posts: 6836
Joined: Thu 23 Mar, 2006 21:36
Location: Israel
Has thanked: 41 times
Been thanked: 530 times

Remove duplicated lines in editor

Post by Shmuel »

KlepetoX wrote: Tue 11 Oct, 2022 15:13 but I need to keep last duplicated line and delete all before
I will probably add such an option. Either today or in the nearest days.
User avatar
HaRT
Moderator
Posts: 10863
Joined: Tue 30 Aug, 2005 17:21
Has thanked: 221 times
Been thanked: 358 times

Remove duplicated lines in editor

Post by HaRT »

KlepetoX, alternatively, you can use the Filters plugin to replace your editor text with the result of running gawk (newer version gets installed e.g. with Git for Windows) with the following script: like this: path\to\gawk.exe -f path\to\uniq.awk -v keep=last
Фар есть инструмент, а не нянька. © 2009 DrKnS
KlepetoX
Posts: 129
Joined: Sun 08 Jan, 2012 08:23
Location: Czech Republic
Has thanked: 2 times
Been thanked: 1 time

Remove duplicated lines in editor

Post by KlepetoX »

Shmuel wrote: Tue 11 Oct, 2022 15:19
KlepetoX wrote: Tue 11 Oct, 2022 15:13 but I need to keep last duplicated line and delete all before
I will probably add such an option. Either today or in the nearest days.
That would be great! Surely not only for me.
Shmuel
Posts: 6836
Joined: Thu 23 Mar, 2006 21:36
Location: Israel
Has thanked: 41 times
Been thanked: 530 times

Remove duplicated lines in editor

Post by Shmuel »

You can download it already (dupfighter-v12.7z)
KlepetoX
Posts: 129
Joined: Sun 08 Jan, 2012 08:23
Location: Czech Republic
Has thanked: 2 times
Been thanked: 1 time

Remove duplicated lines in editor

Post by KlepetoX »

Hart:

Maybe I'll try it, but in the meantime I'll wait for Shmuel solution. I have also my bad own solution - read lines from text file to SQL table from end to beginning, then SELECT DISTINCT to a new table and write to text file back from the end to beginnig again. Complicated, working but not optimal solution outside of FAR.
Shmuel
Posts: 6836
Joined: Thu 23 Mar, 2006 21:36
Location: Israel
Has thanked: 41 times
Been thanked: 530 times

Remove duplicated lines in editor

Post by Shmuel »

KlepetoX, as I wrote in my previous message: it is ready, you can download it.
KlepetoX
Posts: 129
Joined: Sun 08 Jan, 2012 08:23
Location: Czech Republic
Has thanked: 2 times
Been thanked: 1 time

Remove duplicated lines in editor

Post by KlepetoX »

Shmuel:

Thank you, it's not the first time you've helped me and I really appreciate it. Everything works!
Shmuel
Posts: 6836
Joined: Thu 23 Mar, 2006 21:36
Location: Israel
Has thanked: 41 times
Been thanked: 530 times

Remove duplicated lines in editor

Post by Shmuel »

KlepetoX, you're welcome!
Post Reply

Return to “Support and Troubleshooting”