Debian

Koha on Debian Lenny + Squeeze mixed HOWTO

From Koha.org:

Koha is a full-featured open-source ILS. Developed initially in New Zealand by Katipo Communications Ltd and first deployed in January of 2000 for Horowhenua Library Trust, it is currently maintained by a team of software providers and library technology staff from around the globe.

This HOWTO will guide you though Koha 3.0.x installation on Debian Lenny + Squeeze mixed environment (Why in Debian mixed but not stable only? Well... This is my normal procedure with 3rd party tarball package...)


Exim4-LDAP 0.0.7 released

Exim4-LDAP 0.0.7 is a development release, which fix a lot of hidden bugs, more comprehensive clone of Qmail-LDAP individual account features, better documentation and coding syntax, and even more. I am confident that the code is stable enough for wider testing by the community, download now.

Spotlight changes include:

  • Add support with Qmail-LDAP accountStatus attribute.
  • Add support with Qmail-LDAP deliveryProgramPath attribute.
  • Activate home_directory in transport with relative patch support.
  • Add LDAP_MAILROOT support if homeDirectory is not absolute.
  • Debug program pipe transport.
  • Add sample LDIF for demo user demo@example.com.
  • Document installation guideline in INSTALL.

Please refer to /usr/share/doc/exim4-ldap/changelog.gz for complete changelog. Changes since 0.0.5:


Exim4-LDAP 0.0.5 initial released

Exim4-LDAP is my latest development progress. It is a Debian package that depends on exim4-heavy-daemon and slapd, which utilize Qmail-LDAP schema as backend database schema for seamless migration. Unlink Qmail-LDAP which required for manual source patching and compile, Exim4-LDAP just define additional authentication/router/transfer rules within configuration files. The ultimate goal of this research project is going to integrate with Samba-LDAP-PAM (for both Windows/Linux Single-Sign-On), RADIUS (for other services SSO) and even other services such as webmail. The project is now hosting in SourceForge.net with .deb available, download now.

Here I will draft some simple installation guideline which not yet included in source package document. I am now testing this with a newly installed Debian Lenny sandbox.


Configurate OpenLDAP in mirror mode replication

I am going to share some user account & directory with NFS among 2~3 servers. Traditionally we can use NFS+NIS solution, but I would like to try out NFS+LDAP+PAM/NSS plus OpenLDAP mirror mode replication.

For LDAP + PAM/NSS, please refer to my other article: LDAP + Samba PDC + PAM/NSS on Debian Lenny HOWTO

Add OpenLDAP mirror mode replication to existing server

Refer to OpenLDAP Software 2.4 Administrator's Guide: Replication, just add the following EXTRA setup to /etc/ldap/slapd.conf:

# Global section
serverID        1
loglevel        sync stats
moduleload      syncprov
rootdn          "cn=admin,dc=example,dc=com"
rootpw          CHANGE

# syncprov specific indexing
index entryCSN                          eq
index entryUUID                         eq

# syncrepl Provider for primary db
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100

# syncrepl directive
syncrepl      rid=001
              provider=ldap://server2.example.com
              bindmethod=simple
              binddn="cn=admin,dc=example,dc=com"
              credentials=CHANGE
              searchbase="dc=dc=example,dc=com"
              schemachecking=on
              type=refreshAndPersist
              retry="60 +"

mirrormode on

Copy & paste this setup to your server2, change the serverID and provider accordingly.

My suggested procedure for deploy

  1. Shutdown both master/slave OpenLDAP server with:
    /etc/init.d/slapd stop
  2. Backup existing LDAP database under home directory:
    slapcat > ~/master.ldif
  3. Transfer the master.ldif to slave server.
  4. Transfer the master /etc/ldap/slapd.conf to slave server, and update it accordingly.
  5. Flush and import existing LDAP database in slave server:
    rm -rf /var/lib/ldap/*
    slapadd -l ~/master.ldif
    slapindex
    chown -Rf openldap:openldap /var/lib/ldap
  6. Start master server, and then slave server with:
    /etc/init.d/slapd start

My main tips: Always BACKUP before critical action :D

Debug the replication

As we turn on the syslog of slapd, we can keep trace it with:

tail -f -n 50 /var/log/syslog | grep slapd

Try to update some record from either master/slave server, check the log, and also browse the replication result in other server. Done :D


Update my 3ware 9650SE firmware + driver + software

I just figure out that 3ware had a new release of firmware and tools for my 9650SE RAID controller. BTW, as Debian unofficial APT is now legacy without update, I search for package original author webiste: http://jonas.genannt.name/

P.S. According to release note, need to upgrade driver and software BEFORE (prior) upgrade the firmware:

Important: Older drivers and software are not compatible with this release
Drivers and software released prior to 9.5.2 are not fully compatible with the firmware in
this release. Please upgrade to a 3ware driver and software that is 9.5.2 (or newer) prior
to updating the firmware on your 9690SA and/or 9650SE controllers.

Upgrade driver and software

Just simply add the following line to /etc/apt/sources.list:

And import key with:

Run aptitude, update package list, search package and install/upgrade. Done :D

Upgrade firmware

Download the firmware from the following link:
http://www.3ware.com/download/Escalade9690SA-Series/9.5.2/9.5.2-9650-Upg...


Rescuing rm-ed file from EXT3

Background: I run multiple VirtualBox client within a single server. Each VirtualBox client coming with its *.vdi file for sure. Accidentally I run rm to them during migration and backup, and so some *.vdi are lose. I need to rescue them.

Most important action for rescue: unplug the server power and shut-it-down IMMEDIATELY! Try not to preform ANY write action to the target harddisk ANYMORE!

Next, I googling and figure out that foremost may give me a hand. So I will need a Ubuntu in Live-CD and boot up a system for rescue besides loading my target drive directly. Soon after that I figure out ext3grep and seems much better for my target.

foremost

I am now running the scan and hopefully it can get my data back. Here are some procedures about the work:

  1. Unplug the power of server IMMEDIATELY!
  2. Get an Ubuntu Live-CD. I am using 9.04 Desktop edition. We don't care about the version.
  3. Boot the system with Live-CD.
  4. After login, first update aptitude package list with:
    aptitude update
  5. Now install foremost and sshfs (for remote backup):
    apt-get install foremost sshfs
  6. Reference to Tutorial: All about VDIs and figure out its header format (I use hexdump to correct *.vdi for the correct header). Manually add the *.vdi extension to /etc/foremost.conf, e.g.:
    vdi y 137438953472    \x3c\x3c\x20\x3c\x75\x53\x20\x6e       \x00\x00\x00\x00\x00\x00\x00\x00
  7. Mount your remote drive and prepare for the restore.
  8. Run foremost as example:
    cd dir_where_you_mount_your_remote_drive_with_sshfs
    foremost -d -v -T -i /dev/sda3
  9. Go for a lunch or bath or coffee break or something else. Remember that foremost is now scanning the WHOLE drive and so it will take for a VERY long time!

ext3grep

Well... I am now trying to recover as newbie... Hopefully it can help:

ext3grep /dev/sda3 --restore-file home/groups/filename_to_be_restore

I am also seeking for other solutions, too. If you have any idea please let me know :S

UPDATE (20090731): Finally ext3grep is functioning and all files are successfully recovered. As they are *.vdi, after restore and bootup VirtualBox client, it is recommened to run fsck for every drives. To force fsck during system boot, run the following command and reboot the Linux:

touch /forcefsck


Protect your Apache from spam IP attack

Besides mod_evasive, there exists another spam deference module for Apache2 in Debian unstable - libapach2-mod-spamhaus.

Again mod-spamhaus is a configuration-free package. Just install it with apt-get:

apt-get -t unstable install libapache2-mod-spamhaus

Double check the installation with:

find /etc/apache2 -name '*spamhaus*'

Also remember to restart Apache in order to get it active:

/etc/init.d/apache2 restart

P.S. Note that this package is now only available in Debian unstable.


Temperary Exim4 SASL AUTH solution with LDAP-Samba-PAM/NSS + Courier authpam module

Recall to my previous Exim4 + Courier + SSL on Debian etch mini-HOWTO, Exim4 authentication should work fine without enable plain_courier_authdaemon and login_courier_authdaemon support. But case will become a bit more complicated when using LDAP-Samba-PAM/NSS setup: LDAP user will now not able to be authenticated. Why and what's up!?

Why not functioning?

By default Debian's Exim4 already coming with AUTH PLAIN and AUTH LOGIN setup with direct query on /etc/passwd or /etc/shadow as follow (Beware! This is completely not equal as authenticate with PAM!):

plain:
  driver = plaintext
  public_name = PLAIN
.ifndef AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS
  client_send = "<; ${if !eq{$tls_cipher}{}\
                    {^${extract{1}{:}{PASSWDLINE}}\
                     ^${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}\
                   }fail}"
.else
  client_send = "<; ^${extract{1}{:}{PASSWDLINE}}\
                    ^${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}"
.endif

login:
  driver = plaintext
  public_name = LOGIN
.ifndef AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS
  # Return empty string if not non-TLS AND looking up $host in passwd-file
  # yields a non-empty string; fail otherwise.
  client_send = "<; ${if and{\
                          {!eq{$tls_cipher}{}}\
                          {!eq{PASSWDLINE}{}}\
                         }\
                      {}fail}\
                 ; ${extract{1}{::}{PASSWDLINE}}\
                 ; ${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}"
.else
  # Return empty string if looking up $host in passwd-file yields a
  # non-empty string; fail otherwise.
  client_send = "<; ${if !eq{PASSWDLINE}{}\
                      {}fail}\
                 ; ${extract{1}{::}{PASSWDLINE}}\
                 ; ${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}"
.endif

As LDAP users information are now NOT stored within /etc/passwd and /etc/shadow, for sure that above setup will not function (because the PASSWDLINE don't contain such information).

Possible solutions

exim4-auth-001.pngexim4-auth-001.png

Some possible solutions:

  1. Direct authenticate with LDAP backend (reference: http://www.wlug.org.nz/EximSmtpAuth)
  2. Manual setup Exim4 with PAM authentication, as LDAP + PAM/NSS function correctly (reference: http://www.wlug.org.nz/EximSmtpAuth)
  3. Enable Exim4's plain_courier_authdaemon and login_courier_authdaemon support, as Courier's authpam module function correctly (which also means LDAP + PAM/NSS function correctly)

Each setup come with different PROS/CONS:

  1. Direct LDAP authentication: We can even store more information within LDAP, e.g. quota, vocation message, redirect, alias and so on, therefore enrich Exim4 functionality; BTW, this method is the most ideal but complicated in setup.
  2. Manual PAM authentication: A bit simple than above but only able to query authenticate information, and nothing else. There is no default Debian's reference setup, too.
  3. Authenticate though Courier: Most simple as Debian already handle most reference setup, what we only need to do is enable it.

Quick-and-dirty solution

Well... Long story short, as a quick and dirty solution, just enable Exim4's Courier authenticate section as below:

# Authenticate against courier authdaemon

# This is now the (working!) example from
# http://www.exim.org/eximwiki/FAQ/Policy_controls/Q0730
#
Possible pitfall: access rights on /var/run/courier/authdaemon/socket.
plain_courier_authdaemon:
  driver = plaintext
  public_name = PLAIN
  server_condition = \
    ${extract {ADDRESS} \
              {${readsocket{/var/run/courier/authdaemon/socket} \
              {AUTH ${strlen:exim\nlogin\n$auth2\n$auth3\n}\nexim\nlogin\n$auth2\n$auth3\n} }} \
              {yes} \
              fail}
  server_set_id = $auth2
  .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS
  server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}}
  .endif

login_courier_authdaemon:
  driver = plaintext
  public_name = LOGIN
  server_prompts = Username:: : Password::
  server_condition = \
    ${extract {ADDRESS} \
              {${readsocket{/var/run/courier/authdaemon/socket} \
              {AUTH ${strlen:exim\nlogin\n$auth1\n$auth2\n}\nexim\nlogin\n$auth1\n$auth2\n} }} \
              {yes} \
              fail}
  server_set_id = $auth1
  .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS
  server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}}
  .endif

And then ensure Courier is now authenticate with authpam:

authmodulelist="authpam"

Finally give access to Exim4 in order to query Courier authdaemon socket:

chmod 755 /var/run/courier/authdaemon

That's all :D


Exim4 SMTP relay though smarthost without TLS support on Debian Lenny

Similar as my previous approach with GMail SMTP smarthost relay, now I try to use my own ISP's SMTP as relay server. BTW, as HGC Broadband's SMTP server only allow port 25 with plain text authentication, with default Debian setup I get the following error message:

2009-04-02 13:39:56 1LpFee-00055Y-1N ** someotherelse@gmail.com R=smarthost T=remote_smtp_smarthost: SMTP error from remote mail server after MAIL FROM:<someone@example.com> SIZE=45837: host mail02.hgcbroadband.com [210.0.255.175]: 530 Authentication required

This is happened even setup correctly with dpkg-reconfigure exim4-config and /etc/exim4/passwd.client. Some important note from Debian configuration file /etc/exim4/conf.d/auth/30_exim4-config_examples:

# Because AUTH PLAIN and AUTH LOGIN send the password in clear, we
# only allow these mechanisms over encrypted connections by default.
# You can set AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS to allow unencrypted
# clear text password authentication on all connections.

So edit /etc/exim4/conf.d/main/03_exim4-config_tlsoptions and add this line to the top of file:

AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS = true

Run dpkg-reconfigure exim4-config again and try to send email though HGC Broadband's SMTP server, all done :D


Using Exim4 to send Messages through GMail on Debian Lenny

NOTE: For complete and latest version please refer to http://wiki.debian.org/GmailAndExim4

Background

My email server seems a bit crazy on these day: both GMail and Yahoo! Mail block a direct send from my server because of following reason:

  hswong3i@gmail.com
    SMTP error from remote mail server after end of data:
    host gmail-smtp-in.l.google.com [209.85.143.114]:
    550-5.7.1 [x.x.x.x] The IP you're using to send mail is not authorized to
    550-5.7.1 send email directly to our servers. Please use the SMTP relay at your
    550-5.7.1 service provider instead. Learn more at            http://mail.google
    550 5.7.1 .com/support/bin/answer.py?answer=10336 d4si2957764tib.28

When checking http://mail.google.com/support/bin/answer.py?answer=10336 it give me the following reason:

In order to prevent spam, Gmail refuses mail when the sending IP address does not match the sending domain. To send mail from your server to Gmail, we suggest using the SMTP relay provided by your ISP. Please note that we are unable to whitelist IP addresses or otherwise make exceptions at this time.

First of all I guess this is due to the use of dynamic IP, with only forward DNS record (A/MX/CNAME) but no reserve record (PTR). But even update my own DNS server with reserve record, no one else are able to discover it though internet. So I have no way to prove my IP is under my control, and so can't prove myself as not spammer.

Secondly, I try to use SMTP relay from my own ISP. BTW, as I am using HGC broadband my username and password is required for the connection. Ok, this is another issue and not my cup of tea :S

Finally, I try to configure my Exim4 to send email though my own Gmail account. This should be most simple because it will no longer depend on my internet connection, and so I can change my ISP whenever and whatever. Here are some procedure for Debian Lenny.

Procedure for Debian Lenny

Run dpkg-reconfigure exim4-config with following options:

  1. General type of mail configuration: mail sent by smarthost; received via SMTP or fetchmail
  2. System mail name: Your SMTP FNDQ, e.g. smtp.example.com
  3. IP-addresses to listen on for incoming SMTP connections: left as empty
  4. Other destinations for which mail is accepted: Some FNDQ of your domain, e.g. example.com, *.example.com
  5. Domains to relay mail for: Other server of your domain, e.g. *.example.com
  6. Machines to relay mail for: Your local subnet, e.g. 192.168.0.0/24
  7. IP address or host name of the outgoing smarthost: smtp.gmail.com::587
  8. Hide local mail name in outgoing mail? No
  9. Keep number of DNS-queries minimal (Dial-on-Demand)? No
  10. Delivery method for local mail: Maildir format in home directory
  11. Split configuration into small files? Yes

Next, edit /etc/exim4/passwd.client and add the following lines (Note: change information accordingly):

gmail-smtp.l.google.com:yourAccountName@gmail.com:y0uRpaSsw0RD
*.google.com:yourAccountName@gmail.com:y0uRpaSsw0RD
smtp.gmail.com:yourAccountName@gmail.com:y0uRpaSsw0RD

Finally, run update-exim4.conf and everything should work fine (You may also run dpkg-reconfigure exim4-config again and double check every setup). Try to send an email to somewhere else and check your log from /var/log/exim/mainlog.

Restriction

Well... GMail SMTP server will rewrite my mail sender (e.g. someone@example.com) into my GMail account (e.g. someotherelse@gmail.com)... This is not what I am really asking for... Well... So I will fallback to use ISP SMTP relay solution... Most likely they are working in same idea :D

Some other tips

Remember that Gmail is using port 587 + TLS for SMTP relay. Always remember to check if TLS support enabled if you face any error.


Syndicate content