Technical References - [convertchars]

[url], [input], and [convertchars] assure proper formatting and interpretation of text

numero = 183
interpreted = N
texte = [convertchars db=somedb.db][value][/convertchars] Convertchars is a useful context for converting characters, and especially valuable because it gives you the additional ability of defining your own conversions. It is used primarily for displaying your database values on the browser page, and will make sure upper ASCII characters display properly. Used without a specified database, it will convert a short list of common characters, such as curly quotes into straight quotes, and < and > into their html counterparts (to prevent people from forcing html code into forms). The conversions are pulled from the StandardConversions.db in the globals folder (you can amend this database). But its secondary use is to clean data. I have created several databases for specific purposes. For instance, I have one called allowhtml.db which does not convert the < and > characters; another called noillegals.db for stripping out spaces, &, # and a host of other characters people sometimes use in filenames. A conversion database can be called anything and located anywhere, but must have these two specified fields: from and to. Example: 
Item Description: [convertchars][desc][/convertchars][text]cleanfilename=[convertchars db=noillegals.db][filename][/convertchars][/text](used as part of an uploading process)
Example using a table: 
[table name=t1&fields=from,to]é        eè        eê        eë        eû        uù        uü        uî        ií        iï        ió        oü        uÚ        UÓ        OÍ        Iñ        nÑ        NÉ        EÈ        Eç        cà        aÁ        Aà        aá        aÊ        EÀ        A[/table]
The word "café" would be converted into "cafe" using: 
[convertchars table=t1]café[/convertchars]
ParameterDescription
db(Required) path to conversion database which contains list of "from" and "to" conversions of words to other words
tableIn place of a db file, you can specify a named reference to a WebDNA table object.
case(Optional) T or F to indicate that word comparisons should be case-sensitive or not. Default is F, case-insensitive
word(Optional) SS, WW, SW to indicate that words should be matched as SubString, Whole Word, or Start of Word, same as [search] parameters (word breaks subchapter) when matching text in a database
delimiters New for WebDNA 5.1
(Optional) List of single characters which define word boundaries.
Some handy uses for [convertwords] include removing foul language from online message boards, spelling out acronyms, changing nicknames to full names, inserting hyperlinks, and expanding glossary terms. The following conversion database may give you some ideas: 
-- conversion.db --from       to WebDNA     <a href="www.webdna.us">WebDNA Software Corporation</a>Mike       Michaelfoulword   f******dsupport    <a href="mailto:support@webdna.us">support</a>logo       <img src="companylogo.gif">
Added by Brian Fries: There's a bit more to the issue here than just switching to convertwords, and I think it's worth looking at deeper... [convertchars] is actually perfectly fine with replacing one character with multiple characters. If you want to replace a multiple character string with one or more other characters, then [convertwords] is the tool of choice. [convertchars] does two relevant things differently than [convertwords]: 1) convertchars automatically UNURLs the TO and FROM fields in the table, so "%26" is treated as an "&" character rather than the literal "%26" 2) convertchars treats a "+" in the "to" field like a space There are reasons for these differences. The UNURLing of TO and FROM allows specification of invisible characters that you may need to strip out of or add to a data source. The + to SPACE mapping is a bit more esoteric, but I would bet the internal WebDNA program uses (or maybe "used") the same block of code for interpreting query parameters in a URL - the stuff after the ? - where spaces are mapped to + characters by the browser. For example "http://example.com/thispage.dna?user+name=Jane+Doe" gets a formvariable named "user name" with a value of "Jane Doe". To illustrate convertchars vs. convertwords, I spent time working out these examples... Example 1: 
[table name=t1&fields=from,to]& %26[/table]CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]Results:CHARS: L John & Joanna + JaneWORDS: L John %26 Joanna + Jane

Notes: The & was replaced in both cases, but for convertchars it was replaced with & instead of literally %26

Example 2: 
[table name=t1&fields=from,to]%26 %2526[/table]CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]Results:CHARS: L John %26 Joanna + JaneWORDS: L John & Joanna + Jane

Notes: convertchars treats the "to" value of "%26" as the character "&", and the "from" value of "%2526" as "%26" (after unurling %25 to %) convertwords treated "%26" literally as "%26", so it did not find and replace the & character

Example 3: 
[table name=t1&fields=from,to]+ %252B[/table]CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]Results:CHARS: L%2BJohn%2B&%2BJoanna%2B+%2BJaneWORDS: L John & Joanna %252B Jane

Notes: convertchars treats the "to" value of "+" as the SPACE character, so all spaces were replaced with %2526, unurled to %2B, and the "+" character was not replaced convertwords treated the to and from as entered, replacing "+" with, literally, "%2526"

Example 4: 
[table name=t1&fields=from,to]%2B %252B[/table]CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]Results:CHARS: L John & Joanna %2B JaneWORDS: L John & Joanna + Jane

Notes: convertchars treats the "to" value of "%2B" as the "+" character, so "+" was replaced with %2526, unurled to %2B, and spaces were not replaced convertwords found no literal "%2B" strings to replace

Example 5: 
[table name=t1&fields=from,to]' %27+ %2B# %23% %25& %26! %21" %22[/table]CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]Results:CHARS: L+John+&+Joanna+++JaneWORDS: L John %26 Joanna %2B Jane

Notes: This is Michael's original example (with " + Jane" added to the source string). convertchars treats the "to" value of "+" as the SPACE character, so: all spaces were replaced with %2B, unurled to "+" the "&" character was replaced with %26, unurled back to the "&" character the "+" character was not replaced convertwords treated the to and from as entered, so: spaces were not changed the "&" character was replaced with %26 the "+" character was replaced with %2B So, in this case, the results of convertwords represent the desired outcome

Example 6: 
[table name=t1&fields=from,to]' %2527%2B %252B# %2523% %2525& %2526! %2521" %2522[/table]CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]Results:CHARS: L John %26 Joanna %2B JaneWORDS: L John %2526 Joanna + Jane

Notes: From example 5, the "+" from value was replaced with "%2B" and the from "%" characters were replaced with "%25" convertchars treats the "to" value of "%2B" as the "+" character, so: all spaces were left alone the "&" character was replaced with the desired "%26", after unurling "%25" to "%" the "+" character was replaced with the desired "%2B" convertwords treated the to and from as entered, so: spaces were not changed the "&" character was replaced with %2526 (not the desired %26) the "+" character was not replaced So, in this case, the results of convertchars represent the desired outcome

Example 7: 
[grep search=[url]%20[/url]&replace= ][url]L John & Joanna + Jane[/url][/grep]Results:L John %26 Joanna %2B Jane

Notes: I'm not sure of the specifics of the desired mapping beyond what is shown, but if it is to URL everything except space characters, as it appears, then using the built in URL then reverting %20 to the space character does the trick in one line.

[convertchars db=somedb.db][value][/convertchars]
Convertchars is a useful context for converting characters, and especially valuable because it gives you the additional ability of defining your own conversions. It is used primarily for displaying your database values on the browser page, and will make sure upper ASCII characters display properly. Used without a specified database, it will convert a short list of common characters, such as curly quotes into straight quotes, and < and > into their html counterparts (to prevent people from forcing html code into forms). The conversions are pulled from the StandardConversions.db in the globals folder (you can amend this database).

But its secondary use is to clean data. I have created several databases for specific purposes. For instance, I have one called allowhtml.db which does not convert the < and > characters; another called noillegals.db for stripping out spaces, &, # and a host of other characters people sometimes use in filenames. A conversion database can be called anything and located anywhere, but must have these two specified fields: from and to.

Example:

Item Description: [convertchars][desc][/convertchars]
[text]cleanfilename=[convertchars db=noillegals.db][filename][/convertchars][/text]
(used as part of an uploading process)

Example using a table:

[table name=t1&fields=from,to]
é        e
è        e
ê        e
ë        e
û        u
ù        u
ü        u
î        i
í        i
ï        i
ó        o
ü        u
Ú        U
Ó        O
Í        I
ñ        n
Ñ        N
É        E
È        E
ç        c
à        a
Á        A
à        a
á        a
Ê        E
À        A
[/table]

The word "café" would be converted into "cafe" using:

[convertchars table=t1]café[/convertchars]


ParameterDescription
db(Required) path to conversion database which contains list of "from" and "to" conversions of words to other words
tableIn place of a db file, you can specify a named reference to a WebDNA table object.
case(Optional) T or F to indicate that word comparisons should be case-sensitive or not. Default is F, case-insensitive
word(Optional) SS, WW, SW to indicate that words should be matched as SubString, Whole Word, or Start of Word, same as [search] parameters (word breaks subchapter) when matching text in a database
delimiters New for WebDNA 5.1
(Optional) List of single characters which define word boundaries.


Some handy uses for [convertwords] include removing foul language from online message boards, spelling out acronyms, changing nicknames to full names, inserting hyperlinks, and expanding glossary terms. The following conversion database may give you some ideas:

-- conversion.db --
from       to 
WebDNA     <a href="www.webdna.us">WebDNA Software Corporation</a>
Mike       Michael
foulword   f******d
support    <a href="mailto:support@webdna.us">support</a>
logo       <img src="companylogo.gif">


Added by Brian Fries:
There's a bit more to the issue here than just switching to convertwords, and I think it's worth looking at deeper...

[convertchars] is actually perfectly fine with replacing one character with multiple characters. If you want to replace a multiple character string with one or more other characters, then [convertwords] is the tool of choice.

[convertchars] does two relevant things differently than [convertwords]:

1) convertchars automatically UNURLs the TO and FROM fields in the table, so "%26" is treated as an "&" character rather than the literal "%26"
2) convertchars treats a "+" in the "to" field like a space

There are reasons for these differences. The UNURLing of TO and FROM allows specification of invisible characters that you may need to strip out of or add to a data source. The + to SPACE mapping is a bit more esoteric, but I would bet the internal WebDNA program uses (or maybe "used") the same block of code for interpreting query parameters in a URL - the stuff after the ? - where spaces are mapped to + characters by the browser. For example "http://example.com/thispage.dna?user+name=Jane+Doe" gets a formvariable named "user name" with a value of "Jane Doe".

To illustrate convertchars vs. convertwords, I spent time working out these examples...

Example 1:
[table name=t1&fields=from,to]
& %26
[/table]
CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]
WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]

Results:
CHARS: L John & Joanna + Jane
WORDS: L John %26 Joanna + Jane

Notes:
The & was replaced in both cases, but for convertchars it was replaced with & instead of literally %26



Example 2:
[table name=t1&fields=from,to]
%26 %2526
[/table]
CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]
WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]

Results:
CHARS: L John %26 Joanna + Jane
WORDS: L John & Joanna + Jane

Notes:
convertchars treats the "to" value of "%26" as the character "&", and the "from" value of "%2526" as "%26" (after unurling %25 to %)
convertwords treated "%26" literally as "%26", so it did not find and replace the & character



Example 3:
[table name=t1&fields=from,to]
+ %252B
[/table]
CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]
WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]

Results:
CHARS: L%2BJohn%2B&%2BJoanna%2B+%2BJane
WORDS: L John & Joanna %252B Jane

Notes:
convertchars treats the "to" value of "+" as the SPACE character, so all spaces were replaced with %2526, unurled to %2B, and the "+" character was not replaced
convertwords treated the to and from as entered, replacing "+" with, literally, "%2526"



Example 4:
[table name=t1&fields=from,to]
%2B %252B
[/table]
CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]
WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]

Results:
CHARS: L John & Joanna %2B Jane
WORDS: L John & Joanna + Jane

Notes:
convertchars treats the "to" value of "%2B" as the "+" character, so "+" was replaced with %2526, unurled to %2B, and spaces were not replaced
convertwords found no literal "%2B" strings to replace



Example 5:
[table name=t1&fields=from,to]
' %27
+ %2B
# %23
% %25
& %26
! %21
" %22
[/table]
CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]
WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]

Results:
CHARS: L+John+&+Joanna+++Jane
WORDS: L John %26 Joanna %2B Jane

Notes:
This is Michael's original example (with " + Jane" added to the source string).
convertchars treats the "to" value of "+" as the SPACE character, so:
all spaces were replaced with %2B, unurled to "+"
the "&" character was replaced with %26, unurled back to the "&" character
the "+" character was not replaced
convertwords treated the to and from as entered, so:
spaces were not changed
the "&" character was replaced with %26
the "+" character was replaced with %2B
So, in this case, the results of convertwords represent the desired outcome



Example 6:
[table name=t1&fields=from,to]
' %2527
%2B %252B
# %2523
% %2525
& %2526
! %2521
" %2522
[/table]
CHARS: [convertchars table=t1]L John & Joanna + Jane[/convertchars]
WORDS: [convertwords table=t1]L John & Joanna + Jane[/convertwords]

Results:
CHARS: L John %26 Joanna %2B Jane
WORDS: L John %2526 Joanna + Jane

Notes:
From example 5, the "+" from value was replaced with "%2B" and the from "%" characters were replaced with "%25"
convertchars treats the "to" value of "%2B" as the "+" character, so:
all spaces were left alone
the "&" character was replaced with the desired "%26", after unurling "%25" to "%"
the "+" character was replaced with the desired "%2B"
convertwords treated the to and from as entered, so:
spaces were not changed
the "&" character was replaced with %2526 (not the desired %26)
the "+" character was not replaced
So, in this case, the results of convertchars represent the desired outcome



Example 7:
[grep search=[url]%20[/url]&replace= ][url]L John & Joanna + Jane[/url][/grep]

Results:
L John %26 Joanna %2B Jane

Notes:
I'm not sure of the specifics of the desired mapping beyond what is shown, but if it is to URL everything except space characters, as it appears, then using the built in URL then reverting %20 to the space character does the trick in one line.



Terry Wilson and Brian Fries

DOWNLOAD WEBDNA NOW!

Top Articles:

WebDNA Modules

A list of the currently available modules...

Download WebDNA Applications

WebDNA applications...

Tips and Tricks

A list of user-submitted tips ...

F.A.Q

A compilation of some user's questions...

WebDNA Libraries

A list of available libraries for WebDNA...

Technical Change History

This Technical Change History provides a reverse chronological list of WebDNA changes...

Related Readings:

[hideif]

[HideIf Comparison]Hide This HTML[/HideIf]...

[append]

[append db=base...

ODBC

UNIX: Setting up WebDNA - ODBC - Database Server...

[regex]

[regex] is a full implementation of grep...

[random]

will display a random number...

[SQLexecute]

Executes and persits the results of an SQL statement...