Tips

Bookmarks for Open Source ILS

Submitted by hswong3i on Thu, 2009/10/22 - 16:15

http://koha.org/
http://www.nlc.state.ne.us/netserv/opensource.html
http://www.degreetutor.com/library/managing-expenses/open-source-library
http://www.liblime.com/open-source/why-your-library-needs-open-source

Others:
http://www.micro-logic.com.hk/rico_sls.html
http://www.hkedcity.net/iworld/forum/index.phtml?iworld_id=62


Bookmarks for WinXP installation related

Submitted by hswong3i on Tue, 2009/10/06 - 14:35

Complete guideline:
http://unattended.msfn.org/
http://simonslick.com/Microsoft/Windows/XP/Service%20Pack%203/

Additional driver pack:
http://www.driverpacks.net/
http://forum.driverpacks.net/viewtopic.php?id=1756
http://forum.driverpacks.net/viewtopic.php?id=2264

Hotfixes downloader:
http://wud.jcarle.com/

Onepiece's EN-US Windows XP Post-SP3 All-in-One Update Pack:
http://www.ryanvm.net/forum/viewtopic.php?t=6438

Fix missing wallpaper after sysprep:
http://support.microsoft.com/kb/329677

Integrate everything:
http://www.nliteos.com/
http://www.ryanvm.net/msfn/

Detail procedure (for RAW ISO):

  1. Copy source CD with nLite, w/o SP3.
  2. (Optional) Splitstream SP3 with nLite.
  3. Splitstream Post-SP3 AIO package with nLite (NOTE: NOT with RyanVM).
  4. Splitstream additional drivers with DrivePacks (NOTE: NOT with nLite).
  5. Tweak options and services with nLite, and create ISO.
  6. Test it with VirtualBox.
  7. (Optional, refer to my other HOWTO) Copy I386 to harddisk and start installation with local source.

Detail procedure for installed WindowsXP:

  1. Manually download update packages and install it.
  2. Run Windows Update and check with any missing.
  3. Create SAD DrivePack installer and update with it.


iTalc master snapin for FOG with 7-Zip SFX

Submitted by hswong3i on Fri, 2009/09/04 - 15:44

This article will detail the procedures about create snapin for FOG with 7-Zip SFX, which re-package iTalc installation tarball for deploy iTale master application (IMA). This skill can also apply for deploy other program installation.

Background

I am now handling a computer lab with FOG. All computers are installed with iTalc client for classroom management during lesson. I have create a single OS image for all computers, but there is an exception for teacher's computer: it should install with IMA but not only client. Basically I have 2 choices:

  1. Create 2 different OS image: 1 for all student computer where the other for teacher computer ONLY.
    This will GREATLY increase my work load in order to keep both image sync with latest update, software and setting. This is the most simple but the most silly solution.
  2. Clone all computer with single image, and MANUALLY install the IMA for teacher computer after cloning.
    From some point of view it is suitable because IMA installation require more steps than usual: I will need to copy the private keys for setup VNS connection, and also deploy pre-configured classroom setup so all student computer will appear in IMA. But this is not a good solution, too. I am not able to use schedule cloning in FOG because I will need to preform manual setup later.

So the best solution should be FOG snapin. Create a snapin with silent install, therefore FOG will automatically deploy it to client once it is ready after cloning. We will also able to deploy single snapin to individual computer whenever required. But there exists a down side too: not every installation file support silent install. Therefore I will need to "re-package" the program for my purpose.

Technical detail

This solution will need some basic skill for both 7-Zip and Windows batch scripting. The idea would be:

  1. Prepare all required (both original and additional) files for installation under a directory.
  2. Write a simple batch script with "wrap" the original setup.exe with additional parameter and procedure.
  3. Package the target directory as 7-Zip archive.
  4. Write a simple 7-Zip SFX configuration file for the following step.
  5. Create a 7-Zip SFX archive as final product.

Any down side? Yes, for sure. As this is just a simple "wrapper" of original installer, we will still need to preform some manual setup procedure once the snapin is start. Therefore it is not total "automatically". But the benefit is very simple: you will not need to write a complete installer for re-package, and the procedure is easy for master.

I will comment the benefit of this method as: no additional software installation will be missed (the snapin will remind you), and we don't need to prepare the setup file elsewhere (don't need to keep the CD/setup.exe/etc).

Procedure for IMA re-package

First download iTalc installation tarball from SourceForge and unzip it to somewhere, etc, C:\Download\italc-1.0.9.

Next, copy the exported public key as C:\Download\italc-1.0.9\italc_dsa_key.pub, and start the installation as normal. We will now install IMA as normal. Point the exported public key location as . (a single dot, which means current directory). Before installation complete, click "Save installation settings". It will save the setup procedure as C:\Download\italc-1.0.9\installsettings.xml and so we can reuse it within our own batch script.

We need to private keys for IMA, too. I guess you should already have that with a functional iTalc setup. Copy the keys from C:\Program Files\iTALC\keys to C:\Download\italc-1.0.9\keys. We will copy it to new system with batch script.

(Optional, but strongly suggested) Run iTalc IMA in local computer once, setup the "classroom", and add all required student computers into it. Once complete, copy the setup from C:\Documents and Settings\Administrator\Application Data\iTALC to C:\Download\italc-1.0.9 (both globalconfig.xml and personalconfig.xml). Again, we will copy it to new system with batch script.

Let's create the batch installation script now, e.g. C:\Download\italc-1.0.9\keys\setup.bat:

setup installsettings.xml

mkdir "C:\Program Files\iTALC\keys\private\admin\"
copy keys\private\admin\*.* "C:\Program Files\iTALC\keys\private\admin\"

mkdir "C:\Program Files\iTALC\keys\private\supporter\"
copy keys\private\supporter\*.* "C:\Program Files\iTALC\keys\private\supporter\"

mkdir "C:\Program Files\iTALC\keys\private\teacher\"
copy keys\private\teacher\*.* "C:\Program Files\iTALC\keys\private\teacher\"

mkdir "C:\Program Files\iTALC\keys\public\admin\"
copy keys\public\admin\*.* "C:\Program Files\iTALC\keys\public\admin\"

mkdir "C:\Program Files\iTALC\keys\public\supporter\"
copy keys\public\supporter\*.* "C:\Program Files\iTALC\keys\public\supporter\"

mkdir "C:\Program Files\iTALC\keys\public\teacher\"
copy keys\public\teacher\*.* "C:\Program Files\iTALC\keys\public\teacher\"

mkdir "C:\Documents and Settings\Default User\Application Data\iTalc\"
copy globalconfig.xml "C:\Documents and Settings\Default User\Application Data\iTalc\"
copy personalconfig.xml "C:\Documents and Settings\Default User\Application Data\iTalc\"

mkdir "C:\Documents and Settings\Administrator\Application Data\iTalc\"
copy globalconfig.xml "C:\Documents and Settings\Administrator\Application Data\iTalc\"
copy personalconfig.xml "C:\Documents and Settings\Administrator\Application Data\iTalc\"

You directory should look like this.You directory should look like this.

Run the script once and check the result. The IMA should install with silent mode.

Now package everything with 7-Zip. I do so with the GUI mode, and you may handle this with what you like. Create the 7-Zip archive as C:\Download\italc-1.0.9.7z.

UPDATE (20090907): I just figure out 7-ZIP SFX Maker v2.0 which may simplify the following task. NOTE: We should prepare the target *.7z file in advance so the above step should still useful; from my point of view, the following steps are quite strict forward, too.

Prepare the 7-Zip SFX configure file as C:\Download\italc.txt:

;!@Install@!UTF-8!
Title="italc-1.0.9"
BeginPrompt="Do you want to install the italc-1.0.9?"
RunProgram="setup.bat"
;!@InstallEnd@!

Download 7-Zip extra package and extract it. Copy the 7zSD.sfx to C:\Download.

Now the final step: combine everything with following command:

copy /b 7zSD.sfx + italc.txt + italc-1.0.9.7z italc-1.0.9.exe

Sample output.Sample output.Your directory should now look like this.Your directory should now look like this.

Again, check the product by double click and run it. It should prompt up a message before start, and then everything should go on smoothly.

The message promptedThe message promptedAnd it is now running automatically.And it is now running automatically.

Create snapin in FOG

Once the re-package is ready, we can now create it as FOG snapin. That is very simple:

  1. Create a new snapin package in FOG,
    Create a new snapinCreate a new snapin
  2. Link a host with it,
    Link the snapin with a hostLink the snapin with a host
  3. And deploy it.
    Deploy snapin in host advance taskDeploy snapin in host advance task

The FOG client in target computer will cron check with snapin task, download and start it whenever possible. The rest will just as like as what we test the package within local computer. The installer will start and you should take care of it.

In case of *.msi installer, you should use the following parameters:

  1. Set Snapin Run With: to the path of msiexec.exe (i.e.: c:\windows\system32\msiexec.exe)
  2. Set Snapin Run With Arguments: to /i
  3. Set Snapin Arguments: to /qn

Conclusion

This article guide you though how to repackage software with 7-Zip and simple batch script. It is simple enough so you can apply for any other software, even as simple as starting Firefox installer with FOG snapin. You may also preform more additional task with the batch script, e.g. copy default user setup of Firefox to target computer within FOG snapin, too.

References

http://www.fogproject.org/wiki/index.php?title=FOGUserGuide#Snap-ins
http://www.fogproject.org/wiki/index.php?title=Make_Snapins_Using_FOSS


Configurate OpenLDAP in mirror mode replication

Submitted by hswong3i on Thu, 2009/08/06 - 16:47

I am going to share some user account & directory with NFS among 2~3 servers. Traditionally we can use NFS+NIS solution, but I would like to try out NFS+LDAP+PAM/NSS plus OpenLDAP mirror mode replication.

For LDAP + PAM/NSS, please refer to my other article: LDAP + Samba PDC + PAM/NSS on Debian Lenny HOWTO

Add OpenLDAP mirror mode replication to existing server

Refer to OpenLDAP Software 2.4 Administrator's Guide: Replication, just add the following EXTRA setup to /etc/ldap/slapd.conf:

# Global section
serverID        1
loglevel        sync stats
moduleload      syncprov
rootdn          "cn=admin,dc=example,dc=com"
rootpw          CHANGE

# syncprov specific indexing
index entryCSN                          eq
index entryUUID                         eq

# syncrepl Provider for primary db
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100

# syncrepl directive
syncrepl      rid=001
              provider=ldap://server2.example.com
              bindmethod=simple
              binddn="cn=admin,dc=example,dc=com"
              credentials=CHANGE
              searchbase="dc=dc=example,dc=com"
              schemachecking=on
              type=refreshAndPersist
              retry="60 +"

mirrormode on

Copy & paste this setup to your server2, change the serverID and provider accordingly.

My suggested procedure for deploy

  1. Shutdown both master/slave OpenLDAP server with:
    /etc/init.d/slapd stop
  2. Backup existing LDAP database under home directory:
    slapcat > ~/master.ldif
  3. Transfer the master.ldif to slave server.
  4. Transfer the master /etc/ldap/slapd.conf to slave server, and update it accordingly.
  5. Flush and import existing LDAP database in slave server:
    rm -rf /var/lib/ldap/*
    slapadd -l ~/master.ldif
    slapindex
    chown -Rf openldap:openldap /var/lib/ldap
  6. Start master server, and then slave server with:
    /etc/init.d/slapd start

My main tips: Always BACKUP before critical action :D

Debug the replication

As we turn on the syslog of slapd, we can keep trace it with:

tail -f -n 50 /var/log/syslog | grep slapd

Try to update some record from either master/slave server, check the log, and also browse the replication result in other server. Done :D


Rescuing rm-ed file from EXT3

Submitted by hswong3i on Thu, 2009/07/30 - 13:26

Background: I run multiple VirtualBox client within a single server. Each VirtualBox client coming with its *.vdi file for sure. Accidentally I run rm to them during migration and backup, and so some *.vdi are lose. I need to rescue them.

Most important action for rescue: unplug the server power and shut-it-down IMMEDIATELY! Try not to preform ANY write action to the target harddisk ANYMORE!

Next, I googling and figure out that foremost may give me a hand. So I will need a Ubuntu in Live-CD and boot up a system for rescue besides loading my target drive directly. Soon after that I figure out ext3grep and seems much better for my target.

foremost

I am now running the scan and hopefully it can get my data back. Here are some procedures about the work:

  1. Unplug the power of server IMMEDIATELY!
  2. Get an Ubuntu Live-CD. I am using 9.04 Desktop edition. We don't care about the version.
  3. Boot the system with Live-CD.
  4. After login, first update aptitude package list with:
    aptitude update
  5. Now install foremost and sshfs (for remote backup):
    apt-get install foremost sshfs
  6. Reference to Tutorial: All about VDIs and figure out its header format (I use hexdump to correct *.vdi for the correct header). Manually add the *.vdi extension to /etc/foremost.conf, e.g.:
    vdi y 137438953472    \x3c\x3c\x20\x3c\x75\x53\x20\x6e       \x00\x00\x00\x00\x00\x00\x00\x00
  7. Mount your remote drive and prepare for the restore.
  8. Run foremost as example:
    cd dir_where_you_mount_your_remote_drive_with_sshfs
    foremost -d -v -T -i /dev/sda3
  9. Go for a lunch or bath or coffee break or something else. Remember that foremost is now scanning the WHOLE drive and so it will take for a VERY long time!

ext3grep

Well... I am now trying to recover as newbie... Hopefully it can help:

ext3grep /dev/sda3 --restore-file home/groups/filename_to_be_restore

I am also seeking for other solutions, too. If you have any idea please let me know :S

UPDATE (20090731): Finally ext3grep is functioning and all files are successfully recovered. As they are *.vdi, after restore and bootup VirtualBox client, it is recommened to run fsck for every drives. To force fsck during system boot, run the following command and reboot the Linux:

touch /forcefsck


Protect your Apache from spam IP attack

Submitted by hswong3i on Wed, 2009/07/29 - 20:49

Besides mod_evasive, there exists another spam deference module for Apache2 in Debian unstable - libapach2-mod-spamhaus.

Again mod-spamhaus is a configuration-free package. Just install it with apt-get:

apt-get -t unstable install libapache2-mod-spamhaus

Double check the installation with:

find /etc/apache2 -name '*spamhaus*'

Also remember to restart Apache in order to get it active:

/etc/init.d/apache2 restart

P.S. Note that this package is now only available in Debian unstable.


Temperary Exim4 SASL AUTH solution with LDAP-Samba-PAM/NSS + Courier authpam module

Submitted by hswong3i on Sat, 2009/06/20 - 12:52

Recall to my previous Exim4 + Courier + SSL on Debian etch mini-HOWTO, Exim4 authentication should work fine without enable plain_courier_authdaemon and login_courier_authdaemon support. But case will become a bit more complicated when using LDAP-Samba-PAM/NSS setup: LDAP user will now not able to be authenticated. Why and what's up!?

Why not functioning?

By default Debian's Exim4 already coming with AUTH PLAIN and AUTH LOGIN setup with direct query on /etc/passwd or /etc/shadow as follow (Beware! This is completely not equal as authenticate with PAM!):

plain:
  driver = plaintext
  public_name = PLAIN
.ifndef AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS
  client_send = "<; ${if !eq{$tls_cipher}{}\
                    {^${extract{1}{:}{PASSWDLINE}}\
                     ^${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}\
                   }fail}"
.else
  client_send = "<; ^${extract{1}{:}{PASSWDLINE}}\
                    ^${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}"
.endif

login:
  driver = plaintext
  public_name = LOGIN
.ifndef AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS
  # Return empty string if not non-TLS AND looking up $host in passwd-file
  # yields a non-empty string; fail otherwise.
  client_send = "<; ${if and{\
                          {!eq{$tls_cipher}{}}\
                          {!eq{PASSWDLINE}{}}\
                         }\
                      {}fail}\
                 ; ${extract{1}{::}{PASSWDLINE}}\
                 ; ${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}"
.else
  # Return empty string if looking up $host in passwd-file yields a
  # non-empty string; fail otherwise.
  client_send = "<; ${if !eq{PASSWDLINE}{}\
                      {}fail}\
                 ; ${extract{1}{::}{PASSWDLINE}}\
                 ; ${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}"
.endif

As LDAP users information are now NOT stored within /etc/passwd and /etc/shadow, for sure that above setup will not function (because the PASSWDLINE don't contain such information).

Possible solutions

exim4-auth-001.pngexim4-auth-001.png

Some possible solutions:

  1. Direct authenticate with LDAP backend (reference: http://www.wlug.org.nz/EximSmtpAuth)
  2. Manual setup Exim4 with PAM authentication, as LDAP + PAM/NSS function correctly (reference: http://www.wlug.org.nz/EximSmtpAuth)
  3. Enable Exim4's plain_courier_authdaemon and login_courier_authdaemon support, as Courier's authpam module function correctly (which also means LDAP + PAM/NSS function correctly)

Each setup come with different PROS/CONS:

  1. Direct LDAP authentication: We can even store more information within LDAP, e.g. quota, vocation message, redirect, alias and so on, therefore enrich Exim4 functionality; BTW, this method is the most ideal but complicated in setup.
  2. Manual PAM authentication: A bit simple than above but only able to query authenticate information, and nothing else. There is no default Debian's reference setup, too.
  3. Authenticate though Courier: Most simple as Debian already handle most reference setup, what we only need to do is enable it.

Quick-and-dirty solution

Well... Long story short, as a quick and dirty solution, just enable Exim4's Courier authenticate section as below:

# Authenticate against courier authdaemon

# This is now the (working!) example from
# http://www.exim.org/eximwiki/FAQ/Policy_controls/Q0730
# Possible pitfall: access rights on /var/run/courier/authdaemon/socket.
plain_courier_authdaemon:
  driver = plaintext
  public_name = PLAIN
  server_condition = \
    ${extract {ADDRESS} \
              {${readsocket{/var/run/courier/authdaemon/socket} \
              {AUTH ${strlen:exim\nlogin\n$auth2\n$auth3\n}\nexim\nlogin\n$auth2\n$auth3\n} }} \
              {yes} \
              fail}
  server_set_id = $auth2
  .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS
  server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}}
  .endif

login_courier_authdaemon:
  driver = plaintext
  public_name = LOGIN
  server_prompts = Username:: : Password::
  server_condition = \
    ${extract {ADDRESS} \
              {${readsocket{/var/run/courier/authdaemon/socket} \
              {AUTH ${strlen:exim\nlogin\n$auth1\n$auth2\n}\nexim\nlogin\n$auth1\n$auth2\n} }} \
              {yes} \
              fail}
  server_set_id = $auth1
  .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS
  server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}}
  .endif

And then ensure Courier is now authenticate with authpam:

authmodulelist="authpam"

Finally give access to Exim4 in order to query Courier authdaemon socket:

chmod 755 /var/run/courier/authdaemon

That's all :D


Install Windows XP SP3 from local harddisk with USB flash drive and BartPE

Submitted by hswong3i on Wed, 2009/05/06 - 11:10

Usually we install Windows XP SP3 from CDROM, but sometime you would like to keep the installation file within local harddisk and start with it because of:

  1. Faster installation. Install from local harddisk can greatly increase the installation process from around 40min to 20min.
  2. Install new Windows component without insert installation CD.
  3. Install new hardware without insert installation CD.

Long story short, if you have enough disk space, just free feel and try with this solution. If you are as lazy as me, this will be your cpu of tea :D

Prepare Windows XP SP3 setup files in USB flash drive

Nothing special for this part: 1. Insert both Windows XP SP3 installation CD and your USB flash drive to the PC, 2. Copy and paste all content from CD to somewhere else in your USB flash drive (or just simply copy the I386 folder, you only need it). That's all.

Well... Some more tips: You should use a valid CD key with your valid installation CD, and should verify if they are able to working together. Remember that Microsoft issue license in at least 3 difference version: OEM (e.g. CD coming from Dell, HP, IBM, etc), VL/VOL (Volume License, usually for enterprise or large organization) and Retail (that you are able to buy from retailer directly). Don't mix them up.

Prepare BartPE

Again nothing special: 1. Download BartPE installation file from official site, 2. Install BartPE master program into your PC, and 3. Insert your Windows XP SP3 installation CD and follow the guideline. It is quite simple if you are not requesting extra features.

Once everything are ready, we can now process with next step.

Process pre-installation with BartPE

The following screenshots are taken from a VirtualBox client, but the procedures are verified as valide with real PC:

  1. Insert both BartPE bootable CD and USB flash drive BEFORE system boot. Remember that BartPE don't support USB pnp.
  2. Boot your system with BartPE CD.
  3. Format the local harddisk into NTFS with "A43 File Management Utility"
  4. Copy the prepared I386 folder from USB flash drive to C:
    xpsp3_hda_setup-000.jpgxpsp3_hda_setup-000.jpg
  5. Run the following command and start the pre-installation procedure:
    C:\i386\winnt32.exe /syspart:C: /tempdrive:C: /makelocalsource
  6. Follow the guideline and complete the required steps:
    xpsp3_hda_setup-001.jpgxpsp3_hda_setup-001.jpgxpsp3_hda_setup-002.jpgxpsp3_hda_setup-002.jpgxpsp3_hda_setup-003.jpgxpsp3_hda_setup-003.jpgxpsp3_hda_setup-004.jpgxpsp3_hda_setup-004.jpgxpsp3_hda_setup-005.jpgxpsp3_hda_setup-005.jpgxpsp3_hda_setup-006.jpgxpsp3_hda_setup-006.jpg
  7. Once it complete you can now shutdown the PC.

Remember to unplug both BartPE and USB flash drive BEFORE next system reboot for installation. You don't need them from now.

Process real installation from local harddisk

You should now boot your PC with local harddisk. The Windows XP SP3 installer should start as usual with some trim-down:

xpsp3_hda_setup-007.jpgxpsp3_hda_setup-007.jpgxpsp3_hda_setup-008.jpgxpsp3_hda_setup-008.jpgxpsp3_hda_setup-009.jpgxpsp3_hda_setup-009.jpg

Nothing special during the rest of installation procedure, but you will able to FEEL the speed difference. Enjoy :D

References

http://www.vandomburg.net/installing-windows-xp-from-usb/


Some suggested default setup for new Mailman list

Submitted by hswong3i on Thu, 2009/03/12 - 15:18

All default setup for new Mailman list are located in /usr/lib/mailman/Mailman/Defaults.py, BUT please NEVER touch it and override by using /etc/mailman/mm_cfg.py. Here are some suggested setup for internal usage, just copy and append at the end of /etc/mailman/mm_cfg.py:

DEFAULT_ARCHIVE_PRIVATE = 1
DEFAULT_GENERIC_NONMEMBER_ACTION = 3
DEFAULT_LIST_ADVERTISED = No
DEFAULT_MAX_MESSAGE_SIZE = 0
DEFAULT_MSG_FOOTER = ""
DEFAULT_MSG_HEADER = ""
DEFAULT_REQUIRE_EXPLICIT_DESTINATION = No
DEFAULT_SUBJECT_PREFIX = ""

P.S. Someone like to have subject prefix but someone not. If most of your client are using desktop MUA, e.g. Thunderbird or Outlook, having subject prefix can simplify their filtering rules setup. Always remove header/footer message if some of your client are using legacy MUA.


eGroupWare + smbldap-tools integration with SQL hack

Submitted by hswong3i on Mon, 2009/02/23 - 04:08

By default, smbldap-populate will add groups 'Domain User' with gidnumber 513, and 'Administrators' with gidnumber 544. This can map with eGroupWare's default groups 'Default' and 'Admins'. On the other hand, smbldap-populate will add default administrator account 'root' with uidnumber 0, where it is invalid for eGroupWare (eGroupWare count account id with AUTO_INCREMENT, where starting from 1 and so 0 is invalid). We need some tricks in order to integrate them:


Syndicate content