Drupal

BarCamp Hong Kong 2008: What Can I contribute for it?

barcamp Hong Kong 2008

A good invitation from Groups.Drupal (Hong Kong), about BarCamp Hong Kong 2008. I will try to share my research founding with Drupal 7.x + Enterprise DB backend, e.g. Oracle, DB2 and MSSQL; some tricky skill for integrate 3rd layout design with Drupal; also hope to gather with Hong Kong Drupaler and discover for our possibility.

Are you living in Hong Kong? Are you using or interesting with Drupal? Do you hope to meet more friends and share your idea? Is time to show up, and I will be there waiting for you. Let's meet on tomorrow :D

Wirtting query in reserved word safe: any possible solution?

Reserved word conflict is an always pain for multiple DB backend supported system: Drupal also face this for many years (e.g. http://drupal.org/node/371, since June 30, 2002). This is the most complicated bottleneck for daily maintenance and introduce any new DB backend to Drupal: whenever existing DB vendor change their standard, we need to follow it; whenever a new DB backend introduce, another set of reserved words are coming in, too.

There is many possible solution, e.g. prevent use of ANY reserved words based on requirement of ALL supported database engines (as like as Moodle's implementation); BTW, as this solution will left issue as ALWAYS OPEN, I am not going to discuss within this review.

Another possibility is the use of escape characters, e.g. `` for MySQL, "" for PostgreSQL/Oracle/DB2 and [] for MSSQL. This approach is widely used for many successful projects, e.g. phpmyadmin, phppgadmin, Oracle/DB2/MSSQL EM. The primary consideration is: How to implement this syntax for Drupal, which is simple and elegant for both end user and DB abstraction designer?

For sure that we will need some update for existing Drupal queries syntax, where the ultimate goal is query will finally present as following example before process by PHP database connection (e.g. SELECT batch FROM {batch} WHERE bid = :bid AND token = :token from line 15 of includes/batch.inc, CVS HEAD, rewrite with TNG DB standard). In case of MySQL, after translation it should looks like:

SELECT `batch` FROM `batch` WHERE `bid` = :bid AND `token` = :token

I will discuss different syntax change approach, based on: a) user experience, b) similar to ANSI coding style, c) backward compatibility, d) backend complexity, e) overall performance and f) workload for syntax conversion. A simple marking will be given for each item (e.g. good/functioning: 2; fine/not bad: 1; poor/not functioning: 0) so we can have some simple idea between different proposals.

Here is a quick rundown of my review:

Drupal 6.x + PDO_OCI: finally work together

After put down my multiple database backend research for Drupal among past 4 months, now finally get Drupal 6.x and PDO_OCI working together... It is all about my incorrect pdo_oci driver implementation... The progress is now available in here.

What is the bottleneck?

The main bottleneck of pdo_oci driver implementation is all about size of VARCHAR2 support under AL32UTF8. When working with AL32UTF8, as characters may store up to 2 or 3 bytes, the original maximum 4000 characters support of VARCHAR2 will cut off as 1333 (4000/3). If a single table coming with more than 1 column larger than 1333 characters, an annoying ORA-01461 error message MAY occur. This error message will not trigger when using oci8, but always happened with pdo_oci + Drupal 6.x (Siren 1.x)...

Any tries?

I have try some method to escape from this ORA-01461 error, including:

  1. Reduce most text field size mapping as VARCHAR2(1333), which only left text:big map with VARCHAR2(4000).
  2. Replace some fields as BLOB (e.g. fields storing serialized data) for unlimited storage size support, where restrict some other fields as varchar 256 or even smaller.

But above tries both face some critical error, and so need to rollback:

  1. Some fields (e.g. {menu_router}.access_arguments and {menu_router}.access_arguments) can't use VARCHAR2(1333) only, as they are storing serialized data; BTW, if mapping both to VARCHAR2(4000), ORA-01461 will occur.
  2. BLOB field need special handling (i.e drupal_write_record() and db_decode_blob()) which will complicate the I/O task, therefore not suitable for some special fields (e.g. again {menu_router}.access_arguments and {menu_router}.access_arguments). Since {menu_router} will finally store its rendered result in Drupal's cache system (already using BLOB field for storage), a DUPLICATED BLOB I/O handling result as a very fancy and buggy implementation.

Finally...

Well... Back to the main fantastic problem: as Drupal core hack are share among oci8 and pdo_oci version (backend independent), ~60% code and ~80% Oracle related programming logic are shared among oci8 and pdo_oci, where ~80% PDO related programming logic are also shared among pdo_pgsql and pdo_oci, I switch my focus to debug PHP's pdo_oci driver and my Drupal + pdo_oci driver implementation.

So, what is the problem? Christopher Jones on OPAL give me some hints: I have a wrong transaction handling in case of pdo_oci driver implementation. When working with Oracle + BLOB (or even not with BLOB) a transaction is always required, plus a try {} catch (Exception $e) {} in further more operation. Within my old implementation (trimmed version, share and clone between legacy and other PDO driver):

<?php
  $active_db
->beginTransaction();
 
// Execute the statement.
 
$result = $stmt->execute();
  if (
$result === FALSE) {
   
// Some error handling code.
   
$active_db->rollBack();
    return
FALSE;
  }
 
// Some PDO + BLOB stream I/O handling code.
  // On success, commit the transaction.
 
$active_db->commit();
?>

The correct implementation should be:

<?php
  $active_db
->beginTransaction();
  try {
   
// Execute the statement.
   
$result = $stmt->execute();
   
// Some PDO + BLOB stream I/O handling code.
    // On success, commit the transaction.
   
$active_db->commit();
  }
  catch (
Exception $e) {
   
// Some error handling code.
   
$active_db->rollBack();
    return
FALSE;
  }
?>

Conclusion

Even the research progress is functioning with Drupal 6.x, a lot of effort is still required. As Drupal 7.x will completely switch to PDO, where core database drivers (including MySQL and PostgreSQL) implementation are revamped with OOP, I still need to catch up my missing section among the past 4 months.

The patch for next generation is now RTBC, should I wait for its commit and propose my amendment afterward, or keep hard working and propose the amendment before everything shuttle down?

Is time to kick away IE6

I can't control myself laughing when read though this good article: DIE IE - Down with Internet Explorer. If you are web designer or theme developer as what I am, you should have similar feeling. Especially when I am now hosting 4 Drupal themes in parallel, I hate IE6 even more than else before: the standard CSS implementation are often become NOT compatible in IE6; providing IE6 support become my nightmare ;-(

So what can we do for it? Simply educate other users NOT to use IE6 but other better web browsers, e.g. Firefox, Opera or Safari. Or if you are using Drupal for your personal blog and willing to do so, add a custom block within your blog header region by:

  1. Log into your Drupal blog, and go to "admin/build/block/add".
  2. When create the new custom block, use "Time to upgrade your web browser" as block description, and left block title as blank.
  3. Copy and paste the following code snippet into the block body, and use "PHP code" as input format:
    <!--[if lt IE 7]><div style="margin: .5em; padding: .5em; color: #f00; border: 1px solid #d77; background: #fcc;">The web browser you are using is ancient and extinct... like the dinosaurs. Internet Explorer Sux (6) is a menace to society and every web designers' worst enemy, not to mention being unsecure and prone to collecting malware. It is also not supported by Microsoft anymore. Please upgrade to <a href="http://www.mozilla.com/firefox/" target="_blank">Firefox</a> | <a href="http://www.opera.com/" target="_blank">Opera</a> | <a href="http://www.apple.com/safari/download/" target="_blank">Safari</a> or any other browser that doesn't suck. It only takes a few minutes and you are doing yourself a favor. To read more about why it is a good idea to stop using Internet Explorer altogether <a href="http://themebot.com/articles/tips-and-tutorials/die-ie-down-with-internet-explorer">Click Here</a> and then scroll down to read the article.</div><![endif]-->
  4. Back to "admin/build/block" and you should able to find your newly created block with description "Time to upgrade your web browser". Put it into your page header region (or some elsewhere).

And that's it! Now whenever someone browse your site with IE6, they will be warned with using a suck web browser :P

P.S. Firefox3 will ready for download on June 17, 2008. Are you ready for that? Let's join and set the record for most software downloads in 24 hours :)

P.P.S. Opera 9.5 already released on June 12, 2008, too. I give a quick try for it: it is fast and stable, but RTL support seems a bit broken (which is totally fine for 9.2)...

P.P.P.S. I try IE8 Beta1, too... But it sucks once again... Layouts which are function under both IE6/7 are broken in IE8, unless running it as "Emulate IE7"... Margin/padding are once again render in wrong style, and RTL support is CRAZILY broken... What's the hell of that!? Oh, Beta2 is coming in Auguest!? So let's forget about buggy Beta1 now...

My new Drupal theme project - Multiflex-3

After study, develop and debug for almost a month, I have just publish my 4th theme for Drupal - Multiflex-3 (http://drupal.org/project/multiflex3). It may seems to be so strange that why I do a new fork of existing project (http://drupal.org/project/multiflex37): the main point is I found no way to contribute for that as the original developer seems to be hit-and-run away; on the other hand, I start the work from sketch so that is not much relationship between the old and new project ;-)

The new theme is implemented as fluid width layout, multiple columns, with fancy meta icons, and so on. BTW, it is still far from complete: RTL support is not yet start, and the powerful eyes catch primary menu with 2nd level is not yet complete, and 2nd menu is missing at top right corner, too. Anyway, as I need to use this for my client's project, it should be better to first start the work, then come back for review and revamp afterward :D

Besides the above missing section, I am also thing about how to implement the theme layout as flexible as original design (http://www.1234.info/webtemplates/multiflex3/demo/): it is coming with 5 page layout and 7 header region design! Only layout 1 + 4 and header 4 is now implemented. Maybe add number of sub themes for Drupal? Or implement as example CSS so let user to override default layout based on their needs? Well... Still not have final decision :-|

Drupal theme development with RTL support

I just implement a new theme for my personal blog and publish it in drupal.org: Interactive Media. Similar as my other 2 projects (Contented7 and Internet Services), this new theme also coming with RTL supporting for Drupal 6.x. I test all 3 themes with FF3 Beta5, Opera9.2 and IE7, both LTR and RTL should looks fine. Thanks for Zohar and your site, without your help all this RTL development can't be successful :)

Well, some tips for RTL theme development:

  1. You need to find a friend with a RTL Drupal site for testing. This is very important or else you can't test your progress. A simple FTP access and a drupal account which able to select personal theme should be enough for this purpose.
  2. Remember to test with Opera9.2 and Firefox (at least FF3 Beta5). Because this 2 browser are both cross platform and pass with Acid2 test (http://en.wikipedia.org/wiki/Acid2), their RTL CSS implementation is much correct and looks logical. IE7 may/may not display RTL correctly so don't start your RTL development with it. Even FF2 will display RTL CSS wrongly.
  3. First develop your theme and CSS as compatible with CSS2 standard (http://jigsaw.w3.org/css-validator/), before fine tune it for other browsers. From my point of view, unless include individual browser-specific-hack CSS file for each browser (e.g. for IE6/7 and FF2), your theme can never compatible with ALL well known browsers within single simple code base. If you have a team or a lot of active RTL contributors for your theme development, feel free to work it out; or else just simply indicate your theme's RTL support Acid2 compatible browser ONLY, and let others to contribute.

The development for RTL is not too complicated; so if you have time to consider the others needs, just try to work your Drupal 6.x theme with RTL support. My experience of development time required is around 80%/20% (LTR/RTL), and spend totally 3 days for develop Interactive Media from sketch with reference to my other old themes skeleton :)

A first try with Drupal RTL theme development

When upgrading my 2 existing Drupal theme project for 6.x (Contented7 and Internet Services Theme), I give a try with RTL supporting, too. BTW, I face a critical problem: I am using LTR only and so how can I test if my RTL implementation is correct? Or, is that enough for user? I don't really have idea about this...

It spend me some time to upgrade theme for Drupal 6.x, but I really love the new theme engine, especially when parallel develop with Drupal 5.x. It is more flexible and less complicated for theme developer. Thanks for Drupal 6 :-)

Any solution for solving DB reserved word conflict?

I would like to ask for some support and suggestion for a 6-year-old issue: DB reserved word conflict within Drupal core (http://drupal.org/node/371).

Support multiple databases within core should be one of the possible solution for expand market sharing. BTW, the conflict with DB specific reserved words is an always pain for abstraction layer designer and developer, and so affect the daily work of contribute developers. Drupal has also faced this problem since June 30, 2002.

As Drupal 7.x is now open for public development, this issues is activated once again. Based on research and compare with other OSS web application, there is some possible solutions:

  1. Avoid the use of ALL reserved words within core. This is our traditional approach since 2002; on the other hand, Moodle 1.7+ also apply this solution, which support totally 4 databases: MySQL, PostgreSQL, Oracle and MSSQL (http://docs.moodle.org/en/XMLDB_reserved_words).
  2. Escape ALL potential reserved words with escape characters. Gallery2 is using this solution, and is able to support totally 6 databases: MySQL, PostgreSQL, Oracle, DB2, MSSQL and SQLite (http://www.garfieldtech.com/blog/database-abstraction#comment-446).

Some hybrid suggestions are also proposed, but most of them can't solve this problem perfectly. E.g. Handling the escape of reserved word within each database drivers implementation (slow, complicated, and always miss), only handle the conflict of table name (can't solve the problem), using non-pair-up token as escape character (potentially buggy and not the optimal solution), etc.

Both Moodle and Gallery2 are using ADOdb for DB abstraction, but finally solve the reserved word conflict problem in totally different direction. We can find that what DB abstraction can do for this issues is very limited, and usually require for the help from standardizing core query syntax, and be careful in our daily development.

This issue can't be solved by normal simple patch, and usually need the help from most core developers. We may need to find out the best solution which we are all agreed to follow it.

If you are interested in solving this 6-year-old issue, please refer to http://drupal.org/node/371 for more technical detail. Thanks a lot.

DB support as contribute: is it a good idea?

Abstract

Drupal 6.0 is revamped with Schema API, so what's next for Drupal 7.x? PDO for sure! With this powerful data-access abstraction layer, workload will much reduced for DB abstraction layer designers and developers, and finally benefit our contribute developers and end users.

By the way, together with the decision of Drupal 7.x + PDO, there is also some voice about moving PostgreSQL (and so other potential databases support, e.g. Oracle, DB2, MSSQL, etc) support away from core, but contribute; on the other hand, add official SQLite support into Drupal core, together with MySQL.

Is this really a good idea? Or even if it is possible? As an existing Drupal + PostgreSQL users, what will this affect your daily work? As a potential customer of Drupal + Oracle/DB2/MSSQL/etc, is this a good new for you, or just an evil? I would like to provide some brief idea for you within this article.

A minor progress review for SQLite development

I started my SQLite + Drupal development on yesterday, and now I have some good news: able to complete Drupal 6.x installer (hacked with Siren), face the welcome screen, install all optional module (except forum and search), generate dummy content with devel, and finally pass a AB stress test together with MySQL/PostgreSQL/Oracle. Since I work this out within my personal research project, I would like to give a comparison with ordinary Drupal 6.x CVS HEAD, too.

THE MOST IMPORTANT PROGRESS IS: the shared common code base based on my proposed personal battle targets for D7 is really effective. As mentioned above, I just spend 2 DAYS for this development, including the study of official SQLite documentation and existing D6 research progress. I didn't change any line of existing (hacked) core queries, especially for SQLite, after the successful development for MySQL/PostgreSQL/Oracle within Siren.

Syndicate content