Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
MySQL upgrade made unicode unreadable [Solved]
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
yaman666
Tux's lil' helper
Tux's lil' helper


Joined: 13 Dec 2002
Posts: 117
Location: Chicago, IL

PostPosted: Wed May 24, 2006 9:24 pm    Post subject: MySQL upgrade made unicode unreadable [Solved] Reply with quote

I recently did an emerge -u world and among other things that updated mysql to 5.0.21. Now all mysql data that was stored in utf8 looks like:
Code:
Анна

One thing I noticed is that utf8 flag is no longer used by portage and that in my.cnf
Code:
character-sets-dir=utf
got changed to
Code:
character-sets-dir=/usr/share/mysql/charsets
, but there is no utf8.xml in /usr/share/mysql/charsets either.

Did anyone else encounter this problem after upgrade?

Thanks!


Last edited by yaman666 on Fri May 26, 2006 5:06 pm; edited 1 time in total
Back to top
View user's profile Send private message
steveb
Advocate
Advocate


Joined: 18 Sep 2002
Posts: 4564

PostPosted: Wed May 24, 2006 9:35 pm    Post subject: Reply with quote

Did you dumped the data (before updating) with hexadecimal values for binary fields/data?
What collation did you used in your database/tables/fields before the 5.0.21 release?

cheers

SteveB
Back to top
View user's profile Send private message
yaman666
Tux's lil' helper
Tux's lil' helper


Joined: 13 Dec 2002
Posts: 117
Location: Chicago, IL

PostPosted: Wed May 24, 2006 9:44 pm    Post subject: Reply with quote

Yes, I have backups. However, it seems that data is perfectly fine. It looks exactly like when you try to view unicode webpage in non-unicode encoding in a browser. In fact, I can decode it that way, by using a browser. So I am pretty sure the problem is not with corrupt data, but rather with it being output incorrectly. As if MySQL tries to output unicode in latin1. So I want to tell MySQL that the data is in fact unicode and to display it as unicode.

I am not quite sure what you mean by hexadecimal values and collation. If you mean column type - then it's varchar and text.
Back to top
View user's profile Send private message
steveb
Advocate
Advocate


Joined: 18 Sep 2002
Posts: 4564

PostPosted: Wed May 24, 2006 9:51 pm    Post subject: Reply with quote

Okay... how do you know that the data is wrong? Where do you try to display that UTF8 data? In a webpage? In the console? Where?
Back to top
View user's profile Send private message
yaman666
Tux's lil' helper
Tux's lil' helper


Joined: 13 Dec 2002
Posts: 117
Location: Chicago, IL

PostPosted: Wed May 24, 2006 9:58 pm    Post subject: Reply with quote

Both. In both webpage and console the data is displayed wrong. (You can see it at http://anna.ulyanov.net)
Back to top
View user's profile Send private message
steveb
Advocate
Advocate


Joined: 18 Sep 2002
Posts: 4564

PostPosted: Wed May 24, 2006 10:02 pm    Post subject: Reply with quote

Do you have something like PHPMyAdmin? If so, then could you quickly look there if the data is as well in the same way as on your webpage?
Back to top
View user's profile Send private message
steveb
Advocate
Advocate


Joined: 18 Sep 2002
Posts: 4564

PostPosted: Wed May 24, 2006 10:03 pm    Post subject: Reply with quote

btw: Phuuu... I can read cyrillic. It's long time that I don't write any more in that character set. But I can still read it. Funny... I guess, that some thing you learned in school when you was a kid, will never be forgotten.
Back to top
View user's profile Send private message
yaman666
Tux's lil' helper
Tux's lil' helper


Joined: 13 Dec 2002
Posts: 117
Location: Chicago, IL

PostPosted: Wed May 24, 2006 10:04 pm    Post subject: Reply with quote

No, I prefer command line.

But like I said - everything was fine before upgrade, and now utf8 is not being properly identified. And I have a feeling it has something to do with the dissapearance of utf8 flag from portage and missing utf8.xml from /usr/share/mysql/charsets
Back to top
View user's profile Send private message
yaman666
Tux's lil' helper
Tux's lil' helper


Joined: 13 Dec 2002
Posts: 117
Location: Chicago, IL

PostPosted: Wed May 24, 2006 10:07 pm    Post subject: Reply with quote

steveb wrote:
btw: Phuuu... I can read cyrillic. It's long time that I don't write any more in that character set. But I can still read it. Funny... I guess, that some thing you learned in school when you was a kid, will never be forgotten.

True, hard to forget something like that :)
Back to top
View user's profile Send private message
steveb
Advocate
Advocate


Joined: 18 Sep 2002
Posts: 4564

PostPosted: Wed May 24, 2006 10:18 pm    Post subject: Reply with quote

yaman666 wrote:
No, I prefer command line.

But like I said - everything was fine before upgrade, and now utf8 is not being properly identified. And I have a feeling it has something to do with the dissapearance of utf8 flag from portage and missing utf8.xml from /usr/share/mysql/charsets
No! It has nothing to do with that.

I know it, because I host Gentoo Serbia and they as well do write in cyrillic and all the data is in MySQL 5.0.21 with UTF8 support:
Code:
mail / # mysql --user=root --password=$(cat /root/mysql.pwd) -e "SHOW VARIABLES;"
+---------------------------------+--------------------------------------------------------+
| Variable_name                   | Value                                                  |
+---------------------------------+--------------------------------------------------------+
| auto_increment_increment        | 1                                                      |
| auto_increment_offset           | 1                                                      |
| automatic_sp_privileges         | ON                                                     |
| back_log                        | 50                                                     |
| basedir                         | /usr/                                                  |
| bdb_cache_size                  | 8388600                                                |
| bdb_home                        | /var/lib/mysql/                                        |
| bdb_log_buffer_size             | 524288                                                 |
| bdb_logdir                      |                                                        |
| bdb_max_lock                    | 10000                                                  |
| bdb_shared_data                 | OFF                                                    |
| bdb_tmpdir                      | /tmp/                                                  |
| binlog_cache_size               | 32768                                                  |
| bulk_insert_buffer_size         | 8388608                                                |
| character_set_client            | utf8                                                   |
| character_set_connection        | utf8                                                   |
| character_set_database          | utf8                                                   |
| character_set_filesystem        | binary                                                 |
| character_set_results           | utf8                                                   |
| character_set_server            | utf8                                                   |
| character_set_system            | utf8                                                   |
| character_sets_dir              | /usr/share/mysql/charsets/                             |
| collation_connection            | utf8_general_ci                                        |
| collation_database              | utf8_general_ci                                        |
| collation_server                | utf8_general_ci                                        |
| completion_type                 | 0                                                      |
| concurrent_insert               | 1                                                      |
| connect_timeout                 | 5                                                      |
| datadir                         | /var/lib/mysql/                                        |
| date_format                     | %Y-%m-%d                                               |
| datetime_format                 | %Y-%m-%d %H:%i:%s                                      |
| default_week_format             | 0                                                      |
| delay_key_write                 | ON                                                     |
| delayed_insert_limit            | 100                                                    |
| delayed_insert_timeout          | 300                                                    |
| delayed_queue_size              | 1000                                                   |
| div_precision_increment         | 4                                                      |
| engine_condition_pushdown       | OFF                                                    |
| expire_logs_days                | 0                                                      |
| flush                           | OFF                                                    |
| flush_time                      | 0                                                      |
| ft_boolean_syntax               | + -><()~*:""&|                                         |
| ft_max_word_len                 | 84                                                     |
| ft_min_word_len                 | 4                                                      |
| ft_query_expansion_limit        | 20                                                     |
| ft_stopword_file                | (built-in)                                             |
| group_concat_max_len            | 1024                                                   |
| have_archive                    | YES                                                    |
| have_bdb                        | YES                                                    |
| have_blackhole_engine           | YES                                                    |
| have_compress                   | YES                                                    |
| have_crypt                      | YES                                                    |
| have_csv                        | YES                                                    |
| have_example_engine             | NO                                                     |
| have_federated_engine           | YES                                                    |
| have_geometry                   | YES                                                    |
| have_innodb                     | DISABLED                                               |
| have_isam                       | NO                                                     |
| have_ndbcluster                 | DISABLED                                               |
| have_openssl                    | DISABLED                                               |
| have_query_cache                | YES                                                    |
| have_raid                       | NO                                                     |
| have_rtree_keys                 | YES                                                    |
| have_symlink                    | YES                                                    |
| init_connect                    |                                                        |
| init_file                       |                                                        |
| init_slave                      |                                                        |
| innodb_additional_mem_pool_size | 2097152                                                |
| innodb_autoextend_increment     | 8                                                      |
| innodb_buffer_pool_awe_mem_mb   | 0                                                      |
| innodb_buffer_pool_size         | 16777216                                               |
| innodb_checksums                | ON                                                     |
| innodb_commit_concurrency       | 0                                                      |
| innodb_concurrency_tickets      | 500                                                    |
| innodb_data_file_path           | ibdata1:10M:autoextend:max:128M                        |
| innodb_data_home_dir            |                                                        |
| innodb_doublewrite              | ON                                                     |
| innodb_fast_shutdown            | 1                                                      |
| innodb_file_io_threads          | 4                                                      |
| innodb_file_per_table           | OFF                                                    |
| innodb_flush_log_at_trx_commit  | 1                                                      |
| innodb_flush_method             |                                                        |
| innodb_force_recovery           | 0                                                      |
| innodb_lock_wait_timeout        | 50                                                     |
| innodb_locks_unsafe_for_binlog  | OFF                                                    |
| innodb_log_arch_dir             |                                                        |
| innodb_log_archive              | OFF                                                    |
| innodb_log_buffer_size          | 8388608                                                |
| innodb_log_file_size            | 5242880                                                |
| innodb_log_files_in_group       | 2                                                      |
| innodb_log_group_home_dir       | ./                                                     |
| innodb_max_dirty_pages_pct      | 90                                                     |
| innodb_max_purge_lag            | 0                                                      |
| innodb_mirrored_log_groups      | 1                                                      |
| innodb_open_files               | 300                                                    |
| innodb_support_xa               | ON                                                     |
| innodb_sync_spin_loops          | 20                                                     |
| innodb_table_locks              | ON                                                     |
| innodb_thread_concurrency       | 8                                                      |
| innodb_thread_sleep_delay       | 10000                                                  |
| interactive_timeout             | 28800                                                  |
| join_buffer_size                | 131072                                                 |
| key_buffer_size                 | 268435456                                              |
| key_cache_age_threshold         | 300                                                    |
| key_cache_block_size            | 1024                                                   |
| key_cache_division_limit        | 100                                                    |
| language                        | /usr/share/mysql/english/                              |
| large_files_support             | ON                                                     |
| large_page_size                 | 0                                                      |
| large_pages                     | OFF                                                    |
| license                         | GPL                                                    |
| local_infile                    | ON                                                     |
| locked_in_memory                | OFF                                                    |
| log                             | OFF                                                    |
| log_bin                         | OFF                                                    |
| log_bin_trust_function_creators | OFF                                                    |
| log_error                       | /var/log/mysql/mysqld.err                              |
| log_slave_updates               | OFF                                                    |
| log_slow_queries                | OFF                                                    |
| log_warnings                    | 1                                                      |
| long_query_time                 | 10                                                     |
| low_priority_updates            | OFF                                                    |
| lower_case_file_system          | OFF                                                    |
| lower_case_table_names          | 0                                                      |
| max_allowed_packet              | 1047552                                                |
| max_binlog_cache_size           | 4294967295                                             |
| max_binlog_size                 | 1073741824                                             |
| max_connect_errors              | 999999                                                 |
| max_connections                 | 300                                                    |
| max_delayed_threads             | 20                                                     |
| max_error_count                 | 64                                                     |
| max_heap_table_size             | 16777216                                               |
| max_insert_delayed_threads      | 20                                                     |
| max_join_size                   | 18446744073709551615                                   |
| max_length_for_sort_data        | 1024                                                   |
| max_prepared_stmt_count         | 16382                                                  |
| max_relay_log_size              | 0                                                      |
| max_seeks_for_key               | 4294967295                                             |
| max_sort_length                 | 1024                                                   |
| max_sp_recursion_depth          | 0                                                      |
| max_tmp_tables                  | 32                                                     |
| max_user_connections            | 300                                                    |
| max_write_lock_count            | 4294967295                                             |
| multi_range_count               | 256                                                    |
| myisam_data_pointer_size        | 6                                                      |
| myisam_max_sort_file_size       | 2147483647                                             |
| myisam_recover_options          | OFF                                                    |
| myisam_repair_threads           | 1                                                      |
| myisam_sort_buffer_size         | 25165824                                               |
| myisam_stats_method             | nulls_unequal                                          |
| ndb_autoincrement_prefetch_sz   | 32                                                     |
| ndb_force_send                  | ON                                                     |
| ndb_use_exact_count             | ON                                                     |
| ndb_use_transactions            | ON                                                     |
| ndb_cache_check_time            | 0                                                      |
| net_buffer_length               | 8192                                                   |
| net_read_timeout                | 30                                                     |
| net_retry_count                 | 10                                                     |
| net_write_timeout               | 60                                                     |
| new                             | OFF                                                    |
| old_passwords                   | OFF                                                    |
| open_files_limit                | 4096                                                   |
| optimizer_prune_level           | 1                                                      |
| optimizer_search_depth          | 62                                                     |
| pid_file                        | /var/run/mysqld/mysqld.pid                             |
| prepared_stmt_count             | 0                                                      |
| port                            | 3306                                                   |
| preload_buffer_size             | 32768                                                  |
| protocol_version                | 10                                                     |
| query_alloc_block_size          | 8192                                                   |
| query_cache_limit               | 1048576                                                |
| query_cache_min_res_unit        | 4096                                                   |
| query_cache_size                | 0                                                      |
| query_cache_type                | ON                                                     |
| query_cache_wlock_invalidate    | OFF                                                    |
| query_prealloc_size             | 8192                                                   |
| range_alloc_block_size          | 2048                                                   |
| read_buffer_size                | 258048                                                 |
| read_only                       | OFF                                                    |
| read_rnd_buffer_size            | 520192                                                 |
| relay_log_purge                 | ON                                                     |
| relay_log_space_limit           | 0                                                      |
| rpl_recovery_rank               | 0                                                      |
| secure_auth                     | OFF                                                    |
| server_id                       | 1                                                      |
| skip_external_locking           | ON                                                     |
| skip_networking                 | OFF                                                    |
| skip_show_database              | OFF                                                    |
| slave_compressed_protocol       | OFF                                                    |
| slave_load_tmpdir               | /tmp/                                                  |
| slave_net_timeout               | 3600                                                   |
| slave_skip_errors               | OFF                                                    |
| slave_transaction_retries       | 10                                                     |
| slow_launch_time                | 2                                                      |
| socket                          | /var/run/mysqld/mysqld.sock                            |
| sort_buffer_size                | 524280                                                 |
| sql_mode                        |                                                        |
| sql_notes                       | ON                                                     |
| sql_warnings                    | ON                                                     |
| storage_engine                  | MyISAM                                                 |
| sync_binlog                     | 0                                                      |
| sync_frm                        | ON                                                     |
| system_time_zone                | CEST                                                   |
| table_cache                     | 1024                                                   |
| table_lock_wait_timeout         | 50                                                     |
| table_type                      | MyISAM                                                 |
| thread_cache_size               | 0                                                      |
| thread_stack                    | 196608                                                 |
| time_format                     | %H:%i:%s                                               |
| time_zone                       | SYSTEM                                                 |
| timed_mutexes                   | OFF                                                    |
| tmp_table_size                  | 33554432                                               |
| tmpdir                          | /tmp/                                                  |
| transaction_alloc_block_size    | 8192                                                   |
| transaction_prealloc_size       | 4096                                                   |
| tx_isolation                    | REPEATABLE-READ                                        |
| updatable_views_with_limit      | YES                                                    |
| version                         | 5.0.21                                                 |
| version_bdb                     | Sleepycat Software: Berkeley DB 4.1.24: (May 18, 2006) |
| version_comment                 | Gentoo Linux mysql-5.0.21                              |
| version_compile_machine         | i686                                                   |
| version_compile_os              | pc-linux-gnu                                           |
| wait_timeout                    | 28800                                                  |
+---------------------------------+--------------------------------------------------------+
mail / #   



So I know that this can't be the problem. They had as well that kind of problems, but this was because the MySQL connection they used was NOT set to use utf8.

If you look at this post, then you see, that they have exactly the same problem. But the problem is only, because some monkey dumped the data from MySQL and then restored it wrongly back. New posts in the forum can be easy posted in any character set. MySQL handles that without problems.

What you probably have is a problem with your connection from PHP to MySQL. Is your blog aware of the UTF8 connection you have? Do you know, that older versions of PHP did not set the connection to use UTF8 (you could set it to UTF8, but the connection was always not UTF8). I had that problem as well.

Console: The console is not 100% UTF8 aware. So don't judge on the console. The output there may not always display in the proper way.


cheers

SteveB
Back to top
View user's profile Send private message
steveb
Advocate
Advocate


Joined: 18 Sep 2002
Posts: 4564

PostPosted: Wed May 24, 2006 10:31 pm    Post subject: Reply with quote

BTW: The character sets are now in a table inside MySQL. Look at the database "information_schema". There you will find utf8.
Back to top
View user's profile Send private message
steveb
Advocate
Advocate


Joined: 18 Sep 2002
Posts: 4564

PostPosted: Wed May 24, 2006 10:39 pm    Post subject: Reply with quote

I just now posted a message on the Gentoo Serbia board with cyrillic characters.

When I try to display that message in the console, then I get this:
Code:
mail / # mysql --user=root --password=$(cat /root/mysql.pwd) --batch -e "SET CHARACTER SET 'utf8';SELECT post_text from gentoo_srbija.gentoo_posts WHERE topic_id=606;"
post_text
<br />Ш Ñ            Å  Å¡¾
mail / #


But the data is there:
Code:
mail / # mysql --user=root --password=$(cat /root/mysql.pwd) --batch -e "SET CHARACTER SET 'utf8';SELECT LENGTH(HEX(post_text)) FROM gentoo_srbija.gentoo_posts WHERE topic_id=606;"
LENGTH(HEX(post_text))
1776
mail / #


Setting my console to UTF8 does not solve the problem (console still can't display full UTF8):
Code:
mail / # mysql --user=root --password=$(cat /root/mysql.pwd) --batch -e "SET CHARACTER SET 'utf8';SELECT post_text from gentoo_srbija.gentoo_posts WHERE topic_id=606;"
post_text
<br />Ш ш            Š šdž
mail / #



So... you see. It is not MySQL who has a problem.

cheers

SteveB
Back to top
View user's profile Send private message
yaman666
Tux's lil' helper
Tux's lil' helper


Joined: 13 Dec 2002
Posts: 117
Location: Chicago, IL

PostPosted: Wed May 24, 2006 11:57 pm    Post subject: Reply with quote

Thanks a lot for the replies!

The thing is - blog displayed just fine before the upgrade, and now it won't display properly, which led me to believe this had something to do with mysql. I will play with it and try to manually extract data and see what happens. But it's good to know that my configuration "should" be working. I may try to restore backup as well just in case.
Back to top
View user's profile Send private message
llongi
Retired Dev
Retired Dev


Joined: 15 Apr 2004
Posts: 459
Location: Switzerland

PostPosted: Thu May 25, 2006 11:34 am    Post subject: Reply with quote

The fix should be easy: basically now MySQL defaults to UTF-8 (which is a good thing), at compile time and in my.cnf, that's why there is no more "utf8" USE flag (but there is a "latin1" USE flag if you want to switch back to the old charset, but that's not your case).
Now, PHP before used latin1: always. Now, with switching MySQL over to UTF-8 at compile time, it started using UTF-8 as charset, and here we have the problem:
before: MySQL --> UTF-8, PHP --> latin1
now: MySQL --> UTF-8, PHP --> UTF-8
and this, of course, breaks, at least on the webpages.
To fix this we added charset setting support to PHP's MySQL extensions (only distro that does that), so PHP now reads /etc/mysql/my.cnf too and sets the charset accordingly. So, to revert your situation to what it was before (with PHP being latin1), and fix the problem, just do the following:
a) emerge the newest dev-lang/php versions, 4.4.2-r2 for PHP4.X and 5.1.4 for PHP5.X
b) add the following to /etc/mysql/my.cnf:

Code:

[php-cli]
default-character-set=latin1

[php-cgi-fcgi]
default-character-set=latin1

[php-apache2handler]
default-character-set=latin1


this will force the PHP connections to be latin1, and all should be displayed correctly again!
_________________
Best regards, Luca.
Back to top
View user's profile Send private message
Bojan
Tux's lil' helper
Tux's lil' helper


Joined: 18 Oct 2004
Posts: 145
Location: € Matrix

PostPosted: Thu May 25, 2006 2:26 pm    Post subject: Reply with quote

I have a very similar problem and I'd like to postpone transition to UTF-8 to some later time.

If I understand this correctly, emerging mysql with latin1 use flag set would give exactly the same result and there would be no need to edit my.cnf ???

Would that be OK ???
Back to top
View user's profile Send private message
yaman666
Tux's lil' helper
Tux's lil' helper


Joined: 13 Dec 2002
Posts: 117
Location: Chicago, IL

PostPosted: Thu May 25, 2006 9:09 pm    Post subject: Reply with quote

CHTEKK wrote:

add the following to /etc/mysql/my.cnf:

Code:

[php-cli]
default-character-set=latin1

[php-cgi-fcgi]
default-character-set=latin1

[php-apache2handler]
default-character-set=latin1


this will force the PHP connections to be latin1, and all should be displayed correctly again!


That did the trick, thanks! :-)
Back to top
View user's profile Send private message
wandra
n00b
n00b


Joined: 24 May 2005
Posts: 25

PostPosted: Wed Mar 07, 2007 11:08 am    Post subject: Problem returned again ... but why ? Reply with quote

for almost one year this has worked on my system,
but now, when i recompiled php i have the same problem as one year ago --
destroyed unicode-charaters in php-web- applications ...
Code:

Weder hier noch in meinem Herkunftsland Österreich lebt man auf einer unberührten Insel der Seligen. Zu verflochten sind die Waren, Verkehrs- und Menschströme. Es gibt immer Dinge, die irgendwie in der Luft liegen. Wie heißt es so schön: Dass die drohende globale Erwärmung ein Problem ist, das alle angeht und dessen Lösung immer wichtiger wird, weiß man auch auf La Palma.


has been anything changed again on the gentoo-developer side ?

any idea ... ? PLEASE help ..



CHTEKK wrote:
The fix should be easy: basically now MySQL defaults to UTF-8 (which is a good thing), at compile time and in my.cnf, that's why there is no more

...

b) add the following to /etc/mysql/my.cnf:

Code:

[php-cli]
default-character-set=latin1

[php-cgi-fcgi]
default-character-set=latin1

[php-apache2handler]
default-character-set=latin1


this will force the PHP connections to be latin1, and all should be displayed correctly again!
Back to top
View user's profile Send private message
Joe User
n00b
n00b


Joined: 21 Mar 2004
Posts: 10
Location: Germany

PostPosted: Wed Mar 07, 2007 4:15 pm    Post subject: Reply with quote

Hi,

adding the following undocumented options to the php.ini (PHP-5.2.x) fixed the problem for our community:
Code:

mysql.connect_charset=latin1
mysqli.connect_charset=latin1

_________________
FreeBSD Remote Installation ¤ Gentoo Linux Remote Installation
Back to top
View user's profile Send private message
llongi
Retired Dev
Retired Dev


Joined: 15 Apr 2004
Posts: 459
Location: Switzerland

PostPosted: Wed Mar 07, 2007 5:31 pm    Post subject: Reply with quote

Indeed with the newest PHP releases the way this Gentoo-only addon-feature was handled changed, we now use the php configuration system to set the charsets for the MySQL<->PHP connection. That means any php.ini, or php_admin_value in Apache config, or even ini_set() in the script itself works now.
The three new options are:
Code:

mysql.connection_charset
mysqli.connection_charset
pdo_mysql.connection_charset

As you can see it's individually settable for each MySQL extension too.
Also there should really be no problems migrating, it was throughly tested and the migration is automatical, ie. IF you have anything set in my.cnf, from which PHP previously took it's charset config, it would automatically set the right options in the right php.ini... The only cases in which this might not work is if a) my.cnf is not in /etc/mysql/my.cnf (not supported by us) or b) you didn't correctly update your config (etc-update, etc. --> your fault! ;) ).
Hope people will like how this changed, it really is much easier and logical to use now, and is extremely more flexible.
_________________
Best regards, Luca.
Back to top
View user's profile Send private message
Joe User
n00b
n00b


Joined: 21 Mar 2004
Posts: 10
Location: Germany

PostPosted: Thu Mar 08, 2007 9:47 pm    Post subject: Reply with quote

Hi,

CHTEKK wrote:
Indeed with the newest PHP releases the way this Gentoo-only addon-feature was handled changed, we now use the php configuration system to set the charsets for the MySQL<->PHP connection.


so the Gentoo-only changes/additions to the my.cnf are not necssary (valid) anymore?

BTW: etc-update is a fine tool, but sometimes configs have to be written from scratch and therefor can't be automagically updated by etc-update ;)
_________________
FreeBSD Remote Installation ¤ Gentoo Linux Remote Installation
Back to top
View user's profile Send private message
llongi
Retired Dev
Retired Dev


Joined: 15 Apr 2004
Posts: 459
Location: Switzerland

PostPosted: Thu Mar 08, 2007 10:35 pm    Post subject: Reply with quote

Joe User wrote:
so the Gentoo-only changes/additions to the my.cnf are not necssary (valid) anymore?


No, now you just need to set this kind of data, for PHP, in the php.ini or the other appropriate places for PHP configuration. my.cnf doesn't influence PHP's charsets anymore.

Joe User wrote:
BTW: etc-update is a fine tool, but sometimes configs have to be written from scratch and therefor can't be automagically updated by etc-update ;)


I know, but still it's the only way for us to convey the configuration change to the user and show the feature really changed. :)
_________________
Best regards, Luca.
Back to top
View user's profile Send private message
vad3r
Guru
Guru


Joined: 02 May 2004
Posts: 461
Location: Munich, Germany

PostPosted: Fri Mar 09, 2007 1:30 pm    Post subject: Reply with quote

Hi,

thanks for this Thread because i have the same UTF-8 encoding problem with auf PHP based website. I changed the configs according to this thread but it doesn't work for me :?

Do i have to compile MySQL or php in a specific way??
Back to top
View user's profile Send private message
fisk
n00b
n00b


Joined: 29 Mar 2004
Posts: 63

PostPosted: Fri Mar 09, 2007 2:29 pm    Post subject: Reply with quote

My entire database has become charset gimped, basically everything written in my own language comes out distorted like this:

står framför spegeln på morgonen för att fåfängt rätta

Where it's supposed to be

"står framför spegeln på morgonen för att fåfängt rätta"

I suppose it was stupid of me not to back the database up before I ran a system update, but I didn't know it would botch my database this way.

I tried to edit the php.ini, and the my.cnf as suggested above, but nothing happened, I tried exporting the databases affected by it, and when I open them in an editor, they now have these weird characters instead of the ones that used to be there.

Is this reversible? Or do I now have to scrap everything? =/
_________________
-doh
Back to top
View user's profile Send private message
llongi
Retired Dev
Retired Dev


Joined: 15 Apr 2004
Posts: 459
Location: Switzerland

PostPosted: Fri Mar 09, 2007 4:21 pm    Post subject: Reply with quote

@vad3r: No, nothing needs to be compiled in some special way for this to work, and maybe it's better to talk here than in the bug. ;)

@fisk: Huh? The PHP update only changed the way PHP<->MySQL connections are set charset-wise, it's impossible that it actually changed the data IN your database, unless you also made some major upgrade of MySQL or something like that...
The PHP update should really be no problem, it actually should add the right directives itself to php.ini, and it's very simple how to fix it yourself if it does not:
Before, PHP looked at my.cnf, first the [php-$SAPI] section in there, then the [client] section, if you had set any particular charset stuff in one of those two, then it would use it.
Now, PHP just uses the php.ini setting, so you just have to set the same charset you were using in my.cnf (if you were using any!), and then restart Apache, and it will work exactly like before.
If it doesn't, I'm pretty sure it's not the fault of the new PHP ebuilds.
_________________
Best regards, Luca.
Back to top
View user's profile Send private message
Joe User
n00b
n00b


Joined: 21 Mar 2004
Posts: 10
Location: Germany

PostPosted: Fri Mar 09, 2007 6:02 pm    Post subject: Reply with quote

Hi CHTEKK,

if i understand you correctly, i can drop the [php-$SAPI] sections from my.cnf without any sideeffects, because `emerge php` does not look at them anymore? And if so, will the now also unnecessary corresponding mysql-patches be removed too?

BTW: Thanks for unmasking PHP-5.2.x and Suhosin! No overlay/ebuild-hacking anymore...
_________________
FreeBSD Remote Installation ¤ Gentoo Linux Remote Installation
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum