Tuesday, February 14, 2012

Character Problems

The spesific characters of turkish alphabets ->>i?ü???
couse problems while searching in the Fulltext search.
The letter ? is "I" in the uppercase in our alphabet which is equal to the
English letter "I"
The the other lets does not exists in english alphabet.
so the turkish letters always couse problem for us.
Thanks for any sugestions from now.
What happens if you use the neutral word breaker? I take it you are either
using nvarchar, nchar, or nText; or using the Turkish collation.
Also, what is the error you are getting? Are you getting an error message,
or instead are you not getting any results?
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
"Esref DURNA" <EsrefDURNA@.discussions.microsoft.com> wrote in message
news:268F7F6A-41AD-4FFC-AB76-9FCC7510A4F5@.microsoft.com...
> The spesific characters of turkish alphabets ->>igs
> couse problems while searching in the Fulltext search.
> The letter i is "I" in the uppercase in our alphabet which is equal to the
> English letter "I"
> The the other lets does not exists in english alphabet.
> so the turkish letters always couse problem for us.
> Thanks for any sugestions from now.
>
|||The user could write "?zmir" or "izmir"
These quer?es gets different number of results.
We are using nText .
SQLserver Fulltext engine does not support turkish language so we cant
choise it by the wizard.
Thanks for your help
"Hilary Cotter" wrote:

> What happens if you use the neutral word breaker? I take it you are either
> using nvarchar, nchar, or nText; or using the Turkish collation.
> Also, what is the error you are getting? Are you getting an error message,
> or instead are you not getting any results?
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> "Esref DURNA" <EsrefDURNA@.discussions.microsoft.com> wrote in message
> news:268F7F6A-41AD-4FFC-AB76-9FCC7510A4F5@.microsoft.com...
>
>
|||Esref DURNA,
There is no need to use the Full-Text Indexing Wizard as you can use T-SQL
code to accomplish the same thing, for details see:
http://spaces.msn.com/members/jtkane...&_c=blogp art
Specifically, you can use the following SQL code and change the code to
reference your database and table and then change the nText column to use
the Neutral "Language for Word Breaker":
use pubs
go
IF OBJECTPROPERTY ( object_id('pub_info'),'TableHasActiveFulltextIndex ') = 1
BEGIN
print 'Table pub_info is Full-Text Enabled, dropping Full-Text Index &
Catalog...'
EXEC sp_fulltext_table 'pub_info', 'drop'
EXEC sp_fulltext_catalog 'PubInfo', 'drop'
END
ELSE IF OBJECTPROPERTY (
object_id('pub_info'),'TableHasActiveFulltextIndex ') = 0
BEGIN
print 'Table pub_info is NOT Full-Text Enabled, creating FT Catalog,
Index & Activating...'
EXEC sp_fulltext_catalog 'PubInfo', 'create'
EXEC sp_fulltext_table 'pub_info', 'create', 'PubInfo', 'UPKCL_pubinfo'
-- add 0 as the last parameter for the Neutral wordbreaker.
EXEC sp_fulltext_column 'pub_info', 'pub_id', 'add', 0
EXEC sp_fulltext_column 'pub_info', 'pr_info', 'add', 0
EXEC sp_fulltext_table 'pub_info', 'activate'
END
go
Furthermore, you should insure that you are using the "N prefix" when
referencing text and data in the Unicode datatypes of nText, nChar and
nVarchar, see KB article 239530 (Q239530) "INF: Unicode String Constants in
SQL Server Require N Prefix"
http://support.microsoft.com/default...b;en-us;239530 for more
details.
Thanks,
John
SQL Full Text Search Blog
http://spaces.msn.com/members/jtkane/
"Esref DURNA" <EsrefDURNA@.discussions.microsoft.com> wrote in message
news:E3E505F6-756D-4209-8644-7803497690E4@.microsoft.com...[vbcol=seagreen]
> The user could write "Izmir" or "izmir"
> These queries gets different number of results.
> We are using nText .
> SQLserver Fulltext engine does not support turkish language so we cant
> choise it by the wizard.
> Thanks for your help
> "Hilary Cotter" wrote:
either[vbcol=seagreen]
message,[vbcol=seagreen]
the[vbcol=seagreen]
|||Could u tell the steps which we should follow?
and does the fulltext index will incres 1 time when we change the nText to
Language for Word Breaker?
What will be the basic steps?
And Does this query will return true "N'(izmir == ?zmir)" where the
character "?" is the upper case version of the letter i on our language.
And Thanks your help from now
"John Kane" wrote:

> Esref DURNA,
> There is no need to use the Full-Text Indexing Wizard as you can use T-SQL
> code to accomplish the same thing, for details see:
> http://spaces.msn.com/members/jtkane...&_c=blogp art
> Specifically, you can use the following SQL code and change the code to
> reference your database and table and then change the nText column to use
> the Neutral "Language for Word Breaker":
> use pubs
> go
> IF OBJECTPROPERTY ( object_id('pub_info'),'TableHasActiveFulltextIndex ') = 1
> BEGIN
> print 'Table pub_info is Full-Text Enabled, dropping Full-Text Index &
> Catalog...'
> EXEC sp_fulltext_table 'pub_info', 'drop'
> EXEC sp_fulltext_catalog 'PubInfo', 'drop'
> END
> ELSE IF OBJECTPROPERTY (
> object_id('pub_info'),'TableHasActiveFulltextIndex ') = 0
> BEGIN
> print 'Table pub_info is NOT Full-Text Enabled, creating FT Catalog,
> Index & Activating...'
> EXEC sp_fulltext_catalog 'PubInfo', 'create'
> EXEC sp_fulltext_table 'pub_info', 'create', 'PubInfo', 'UPKCL_pubinfo'
> -- add 0 as the last parameter for the Neutral wordbreaker.
> EXEC sp_fulltext_column 'pub_info', 'pub_id', 'add', 0
> EXEC sp_fulltext_column 'pub_info', 'pr_info', 'add', 0
> EXEC sp_fulltext_table 'pub_info', 'activate'
> END
> go
> Furthermore, you should insure that you are using the "N prefix" when
> referencing text and data in the Unicode datatypes of nText, nChar and
> nVarchar, see KB article 239530 (Q239530) "INF: Unicode String Constants in
> SQL Server Require N Prefix"
> http://support.microsoft.com/default...b;en-us;239530 for more
> details.
> Thanks,
> John
> --
> SQL Full Text Search Blog
> http://spaces.msn.com/members/jtkane/
>
> "Esref DURNA" <EsrefDURNA@.discussions.microsoft.com> wrote in message
> news:E3E505F6-756D-4209-8644-7803497690E4@.microsoft.com...
> either
> message,
> the
>
>
|||The problem is that Izmir is considered to be a different word from "izmir"
as the capital letter I has a diacritic on it. For this to work you will
need a Turkish word breaker; I don't believe one currently exists for SQL
2000, however SQL 2005 does ship with one.
Depending on your requirements you may be able to use SQL 2005 which is
currently in beta 2.
Otherwise you might be able to parse at the client to trap for I and replace
it with I and I. So a search on Izmir would be expanded to Izmir OR Izmir
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
"Esref DURNA" <EsrefDURNA@.discussions.microsoft.com> wrote in message
news:E3E505F6-756D-4209-8644-7803497690E4@.microsoft.com...[vbcol=seagreen]
> The user could write "Izmir" or "izmir"
> These queries gets different number of results.
> We are using nText .
> SQLserver Fulltext engine does not support turkish language so we cant
> choise it by the wizard.
> Thanks for your help
> "Hilary Cotter" wrote:
either[vbcol=seagreen]
message,[vbcol=seagreen]
the[vbcol=seagreen]
|||Esref DURNA,
Sure, I've modified the script from my blog to be more generic and less
specific to the Pubs database, you will need to modify it and replace the
<values> with your real FT Catalog name, table and column names.
I don't understand this question: "does the fulltext index will incres 1
time when we change the nText to Language for Word Breaker?" Could you
explain in more detail what is your concern?
Q. Does this query will return true "N'(izmir == Izmir)" where the character
"I" is the upper case version of the letter i on our language?
A. The answer is both yes and no. Full Text Search queries are
case-insenstive, so normally a lower-case word "izmir" is the equalivant of
"Izmir", but in this case the upper-case letter "I" also has a accented
letter and Full Text Search queries are accent senstive regardless of the
database or column collation. Specificly, for Full Text Search queries
"N'(izmir != Izmir)".
use <your_database_name>
go
--> NOTE: Only run this ONCE per database !!!
sp_fulltext_database 'enable'
go
IF OBJECTPROPERTY (
object_id('<your_table_name>'),'TableHasActiveFull textIndex') = 1
BEGIN
print 'Table <your_table_name> is Full-Text Enabled, dropping Full-Text
Index & Catalog...'
EXEC sp_fulltext_table '<your_table_name>', 'drop'
EXEC sp_fulltext_catalog '<your_FT_Catalog_name>', 'drop'
END
ELSE IF OBJECTPROPERTY (
object_id('<your_table_name>'),'TableHasActiveFull textIndex') = 0
BEGIN
print 'Table <your_table_name> is NOT Full-Text Enabled, creating FT
Catalog, Index & Activating...'
EXEC sp_fulltext_catalog '<your_FT_Catalog_name>', 'create'
EXEC sp_fulltext_table '<your_table_name>', 'create',
'<your_FT_Catalog_name>', 'UPKCL_pubinfo'
EXEC sp_fulltext_column '<your_nText_Column_name>',
'<your_Table_Primary_key', 'add',0
EXEC sp_fulltext_table '<your_table_name>', 'activate'
END
-- After Enabling & Activating Tables, Columns & Indexes - Start Full
Population
BEGIN
SET NOCOUNT ON
DECLARE @.begin datetime
DECLARE @.end datetime
SET @.begin = CURRENT_TIMESTAMP
EXEC sp_fulltext_catalog '<your_FT_Catalog_name>', 'start_full' -- "Full
Crawl"
-- Wait for crawl to complete
DECLARE @.status int, @.itemCount int, @.keyCount int, @.indexSize int
SELECT @.status = FullTextCatalogProperty('<your_FT_Catalog_name>',
'populatestatus')
WHILE (@.status <> 0)
BEGIN
WAITFOR DELAY '00:00:01' -- wait for 1 second before checking FT
Populatestatus...
SELECT @.status = FullTextCatalogProperty('<your_FT_Catalog_name>',
'populatestatus')
END
SET NOCOUNT OFF
END
go
-- Confirm above results with:
SELECT *
FROM <your_Table_name> WHERE CONTAINS(*, '"<search_word>"')
go
Hope this helps!
John
SQL Full Text Search Blog
http://spaces.msn.com/members/jtkane/
"Esref DURNA" <EsrefDURNA@.discussions.microsoft.com> wrote in message
news:93AF07D4-3228-4661-B291-67D38D780CE9@.microsoft.com...[vbcol=seagreen]
> Could u tell the steps which we should follow?
> and does the fulltext index will incres 1 time when we change the nText to
> Language for Word Breaker?
> What will be the basic steps?
> And Does this query will return true "N'(izmir == Izmir)" where the
> character "I" is the upper case version of the letter i on our language.
> And Thanks your help from now
> "John Kane" wrote:
T-SQL[vbcol=seagreen]
http://spaces.msn.com/members/jtkane...&_c=blogp art[vbcol=seagreen]
use[vbcol=seagreen]
= 1[vbcol=seagreen]
&[vbcol=seagreen]
'UPKCL_pubinfo'[vbcol=seagreen]
in[vbcol=seagreen]
message[vbcol=seagreen]
equal to[vbcol=seagreen]
|||hmmmm, the diacritic somehow has disappeared. What I meant to say is trap
for the I with the dot on top of it and then expand your search phrase on
the original word with the dot on top of the I and or it with the plain I,
like this
(Izmir OR I(with a dot on top)zmir) AND (remaining tokens in your search
phrase)
"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:ehqyyT%238EHA.208@.TK2MSFTNGP12.phx.gbl...
> The problem is that Izmir is considered to be a different word from
> "izmir"
> as the capital letter I has a diacritic on it. For this to work you will
> need a Turkish word breaker; I don't believe one currently exists for SQL
> 2000, however SQL 2005 does ship with one.
> Depending on your requirements you may be able to use SQL 2005 which is
> currently in beta 2.
> Otherwise you might be able to parse at the client to trap for I and
> replace
> it with I and I. So a search on Izmir would be expanded to Izmir OR Izmir
>
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> "Esref DURNA" <EsrefDURNA@.discussions.microsoft.com> wrote in message
> news:E3E505F6-756D-4209-8644-7803497690E4@.microsoft.com...
> either
> message,
> the
>

No comments:

Post a Comment