Bsrgvty

Sometimes I need to search files with accented characters (diacritic in general), usually with locate/mlocate. I wish to setup (maybe in /etc/updatedb.conf) so it let me search for this special characters using a certain language mapping, for example:

a == âàáäÂÀÂÄ
e == êèéëÊÈÉË
i == îïíÎÏ
o == ôöóÔÖ
u == ûùüÛÜÙ
c == çÇ
n == ñ

So locate -i liberación also search for files with string liberacion and even liberaciòn.

Notes and assumptions

• And maybe others: ÂÃÄÀÁÅÆ ÇÈÉÊËÌÍÎÏ ÐÑÒÓÔÕÖØÙÚÛÜÝÞ ßàáâãäåæç èéêëìíîïðñòóôõö øùúûüýþÿ.


• This is a common situation on romance languages like Spanish, French and German.


• I'm always using a locale 100% UTF-8.


• I would rather not have to use regular expressions.


• A patch might use ASCII transliterations of Unicode as Unidecode/cUnidecode does. Most of mlocate is written on C.

Related

• Similar question but using find
  


• Miloslav Trmač (mlocate developer) say here that the official source code is on pagure.io (and a fork on Github).


• I file an issue on mlocate repo at Pagure.io to add this feature.
  
  
  • 
    Update 2018-02: This can be fixed with this pull request by marcotrevisan. Will add a -t/--transliterate support using iconv to match accented.
  
  
  • 
    Update 2018-03: mlocate with support for --transliterate is now included in Ubuntu 18.04 LTS Bionic Beaver (v2 and v3.1).
  
  

If we take a look at updatedb.conf(5), we'll find that there is no much we can do with configuration items.

So we are going to write a script using locate; At the end we are able to run something like my-locate.sh liberacion or my-locate.sh liberâciòn and it will brings us all the possible combinations.

Lets start

First create a simple file as our database anywhere you want it to be, e.g: ~/.mydb; then add your accents characters into that file like this:

aâàáäÂÀÂÄ
eêèéëÊÈÉË
iîïíÎÏ
uûùüÛÜÙ
cçÇ
oôöóÔÖóòòò
...
...

Then we need a small script which does the job for us, I wrote a simple one:

#!/bin/bash

# Final search term 
STR=""

# Loop throughout all characters of desired string
for (( i=0; i<$#1; i++ )); do

 # Split the string in one char
 CH="$1:$i:1"

 # Find all possible combinations of this char
 CHARS=$(grep "$CH" ~/.mydb)

 # Add an "or" operator between characters
 REG=$(echo "$CHARS" | sed 's/.1/&|/g' )
 REG="($REG)"

 # Append all possible combination of this character
 # to our final search term as an or statement
 if [ "$REG" == '()' ];
 then
 STR=$STR$CH
 else
 STR=$STR$REG
 fi

done

# locate it using regex
locate --regex "$STR$"

Now save it somewhere in your PATH with a desired name, e.g: in ~/bin. It should be already in your PATH environment.

After all simply use something like this to search all possible combinations.

my-locate.sh liberacion

Will find for me all of these:

~/lab/liberacion
~/lab/liberaciòn
~/lab/liberación
~/lab/liberâciòn
~/lab/liberäciòn
~/lab/libÈrâciòn

Now with mlocate 0.26 we have -t --transliterate option (see the man page) on Ubuntu 18.04+ (without the need of odd workarounds):

Creating some test files:

$ touch liberación liberacion liberaciôn

Update and search:

$ updatedb
$ locate --transliterate liberacion 
/home/pablo/liberacion
/home/pablo/liberación
/home/pablo/liberaciôn

So now locate -t liberación also search for files with string liberacion and even liberaciòn!

Finally, creating an alias on my .bashrc :-)

$ alias locate="locate --transliterate"