Booting Ubuntu Server 10.10 from 3TB Drive

yes that’s right I wanted to upgrade my server from 250GB Drive to a 3TB drive but after the first time around installing Ubuntu server and letting Ubuntu setup my partitions I got this:

no bootable device insert boot disk and press any key

I tried booting into parted to see what was up but everything looked great in the partition table. My BIOS was recognizing the disk as 3TB so I don’t think that was the issue BUT it was…

After a good amount of hours wasted since the format process of a 3TB drive is not something that is done in 2 seconds… I found this little piece of info that told me I have to mark the partition that is booting with a boot flag. I’m running this server on a Intel D510MO board and there is something there where it can’t recognize the grub boot partition unless its marked with the boot flag…

here is how you go about doing the install with glory and wonderful colors!!

Run the Default Ubuntu Server install and choose the default “Let Ubuntu manage and create your partitions” (I picked with LVM) then after the install your server will reboot and you should get the can’t find your disk message. Now boot from the install Ubuntu Server 10.10 disk again and select boot from first disk option!!

This will boot your system and you can now access a shell. Here are the magic commands:

1. Type “fdisk /dev/sda”.
2. Type “p” to view the MBR partition table. It should show one partition, with “ee” in the “Id” column. This is normal, even though you created several GPT partitions.
3. Type “a” and enter “1” (or whatever the partition number is) when prompted.
4. Type “p” again. You should now see an “*” under the “Boot” column.
5. Type “w” to save the changes.

Reboot and watch your system auto boot from your 3TB drive !!! WOHO babe…

Why did you choose MySQL?

I would like to know the reason that you grabbed MySQL and not PostgreSQL? Easy of install? Think that its better? what you know?

Leave a comment with your thoughts…

Mysql 5.5, Snow leopard and rails

I had some issues getting this to work with this error message:

LoadError: dlopen(/Library/Ruby/Gems/1.8/gems/mysql2-0.2.4/lib/mysql2/mysql2.bundle, 9): Library not loaded: libmysqlclient.16.dylib
Referenced from: /Library/Ruby/Gems/1.8/gems/mysql2-0.2.4/lib/mysql2/mysql2.bundle
Reason: image not found – /Library/Ruby/Gems/1.8/gems/mysql2-0.2.4/lib/mysql2/mysql2.bundle
from /Library/Ruby/Gems/1.8/gems/mysql2-0.2.4/lib/mysql2/mysql2.bundle
from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in `require’
from /Library/Ruby/Gems/1.8/gems/mysql2-0.2.4/lib/mysql2.rb:7
from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:36:in `gem_original_require’
from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:36:in `require’
from (irb):1

After a few (hundred) tries I found that you need to change the mysql2.bundle to get this to work, and here is how…

First installing mysql 5.5 from dev.myql.com/downloads using the dmg package… easy as pie….

Second ruby/rails/gems have other issues that I don’t want to go into BUT have rails running then install mysql2 gem using the normal command sudo gem install mysql2

Now is the “tricky” part:

sudo install_name_tool -change libmysqlclient.16.dylib /usr/local/mysql/lib/libmysqlclient.16.dylib /Library/Ruby/Gems/1.8/gems/mysql2-0.2.4/lib/mysql2/mysql2.bundle

This makes a change in the mysql2.bundle file that is not using the full path for libmysqlclient library…

then to test fire up irb

>> require ‘mysql2’
=> true

Securing Apaches Header Information

When you install apache with default settings (./configure && make && make install) this is the header that your users are able to get:

[root@hostname] /usr/local/apache2 $ telnet localhost 80
Trying 127.0.0.1…
Connected to localhost.localdomain (127.0.0.1).
Escape character is ‘^]’.
HEAD / HTTP/1.0
HTTP/1.1 200 OK
Date: Tue, 24 Aug 2010 18:17:56 GMT
Server: Apache/2.2.16 (Unix)
Last-Modified: Sat, 20 Nov 2004 20:16:24 GMT
ETag: “7001c-2c-3e9564c23b600”
Accept-Ranges: bytes
Content-Length: 44
Connection: close
Content-Type: text/html

Now what you could do is go and change your ServerTokens to Prod which is short for ProductOnly and would show only Apache. Like this:

[root@hostname] /usr/local/apache2 $ telnet localhost 80

Trying 127.0.0.1…

Connected to localhost.localdomain (127.0.0.1).

Escape character is ‘^]’.

HEAD / HTTP/1.0

HTTP/1.1 200 OK

Date: Tue, 24 Aug 2010 18:23:39 GMT

Server: Apache

Last-Modified: Sat, 20 Nov 2004 20:16:24 GMT

ETag: “7001c-2c-3e9564c23b600”

Accept-Ranges: bytes

Content-Length: 44

Connection: close

Content-Type: text/html

Much better but you are still telling the world that you are using Apache. First thought would be that you can use mod_headers to just remove the Server: part of your headers but unfortunately this is not possible. Apache tags the Server: and Date: information to the header just before this is sent to the client. There is a bug for this issue but it has not been fixed and it has been around for 5 years.

My fix for this is to create a patch that sets the name “Apache” to whatever you would like. Here is a server that has been patched:

[root@hostname] /usr/local/apache2 $ telnet localhost 80

Trying 127.0.0.1…

Connected to localhost.localdomain (127.0.0.1).

Escape character is ‘^]’.

HEAD / HTTP/1.0

HTTP/1.1 200 OK

Date: Tue, 24 Aug 2010 18:23:39 GMT

Server: Freddys Secure HTTP Server

Last-Modified: Sat, 20 Nov 2004 20:16:24 GMT

ETag: “7001c-2c-3e9564c23b600”

Accept-Ranges: bytes

Content-Length: 44

Connection: close

Content-Type: text/htm

The file you want is Apache_source/include/ap_release.h then look for line 44:

#define AP_SERVER_BASEPRODUCT “Freddys Secure HTTP Server”

If you don’t like messing with the source code here is a patch that you can apply before running the configure… :

diff -rupN httpd-2.2.16/include/ap_release.h httpd-2.2.16_patched/include/ap_release.h
— httpd-2.2.16/include/ap_release.h 2010-07-21 12:26:44.000000000 -0600
+++ httpd-2.2.16_patched/include/ap_release.h 2010-08-24 12:40:09.000000000 -0600
@@ -41,7 +41,7 @@
*/
#define AP_SERVER_BASEVENDOR “Apache Software Foundation”
#define AP_SERVER_BASEPROJECT “Apache HTTP Server”
-#define AP_SERVER_BASEPRODUCT “Apache”
+#define AP_SERVER_BASEPRODUCT “Freddys Secure HTTP Server”

#define AP_SERVER_MAJORVERSION_NUMBER 2
#define AP_SERVER_MINORVERSION_NUMBER 2

Now put this patch in a file called apache_name.patch and run this command:

[root@hostname] ~/httpd-2.2.16 $  $ patch -p1 < ../apache_name.patch
patching file include/ap_release.h

Now like normal run ./configure && make && make install, set your ServerTokens to Prod and the Server: header should say whatever you change the AP_SERVER_BASEPRODUCT too.

How to get a thread-dump from Tomcat or JBoss

Getting a thread-dump is one of the first steps in troubleshooting a misbehaving Java application. There are a few good ways todo this task but the easiest of them all is

kill -3 <PID>

This will send the thread-dump to your stdout logs.

uninitialized constant MysqlCompat::MysqlRes error when starting Rails

After a few upgrades, re-installs and moving to new servers I have been getting this error message uninitialized constant MysqlCompat::MysqlRes when I start rails. (Passenger or Thin)

I have been using mysqlplus gem but wanted to move back to MySQL 2.8.1 gem and I think that is when this started to popup more often. (after looking closer I have servers with ruby 1.8.7 and 1.8.6 using the mysql 2.8.1 gem without issues… )

After firing up thin using strace on my CentOS server I found that for some reason the mysql gem is looking for libmysqlclient.so.16 in /lib64 /usr/lib64 … and not in /opt/mysql/lib where these libraries are located. What I did to fix this issue was to create a link in /usr/lib64/ pointing to the libmysqlclient.so.16 library and voila! the problem is fixed.

PS I think you can set the mysql-lib-dir when you install the mysql gem but I have not tested this yet. I’m thinking something like this might have fixed the issue too(not tested):

[root@server]# gem install mysql -- --with-mysql-dir=/opt/mysql \
                                    --with-mysql-lib=/opt/mysql/lib \
                                    --with-mysql-include=/opt/mysql/include

 

How to generate a self-signed OpenSSL certificate for Apache

There are just a few quick and easy steps to generate a certificate without a passphrase for Apache. First you have to generate a key for your host:

[root@heimdull]# openssl genrsa 1024 > host.key

Generating RSA private key, 1024 bit long modulus
..........................................++++++
.........++++++
e is 65537 (0x10001)
Generating RSA private key, 1024 bit long modulus.............
.............................++++++.........++++++e is 65537 (0x10001)

Now you have your host key file that you will use in the Apache configuration file and to generate the actual certificate

[root@heimdull]# openssl req -new -x509 -nodes -sha1 -days 365 -key host.key > host.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [GB]:GB
State or Province Name (full name) [Berkshire]:Berkshire
Locality Name (eg, city) [Newbury]: Newbury
Organization Name (eg, company) [My Company Ltd]:My Company Ltd
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:www.myserver.com
Email Address []:

Now you have to move these files somewhere that is related to you Apache installation and in your httpd.conf or httpd-ssl.conf file you will need these lines:

SSLEngine on
  SSLCertificateFile /Apache-home/ssl/host.crt
  SSLCertificateKeyFile /Apache-home/ssl/host.key

Installing Glassfish v2.1 and configuring a cluster

I have been a Tomcat guy for the longest time but from time to time I like to play with the other guys software so it was either Jetty or Glassfish. I choose Glassfish. I’ve used Glassfish before for development testing and running rails but have not had any real J2ee apps running inside. Let’s have a look at the installation procedures for Glassfish and what you get after a few minutes of clicking.

Download

Glassfish can be downloaded from https://glassfish.dev.java.net/downloads/v2.1-b60e.html we are going to use the latest stable (production ready) build of v2.1 b60.

Installation

I’m running this on a Linux (Centos) server so here are the steps to install and setup Glassfish WITH the cluster configurations:

[root@heimdull]# java -Xmx256m -jar glassfish-installer-v2.1-b60e-linux-ml.jar
[root@heimdull]# cd glassfish
[root@heimdull]# chmod -R +x lib/ant/bin
[root@heimdull]# lib/ant/bin/ant -s setup-cluster.xml
bla bla bla
....

BUILD SUCCESSFUL
Total time: xx seconds

Configuration

Before we configure some custom settings lets just get this thing started…

[root@heimdull]# export PATH=~/glassfish/bin:$PATH
[root@heimdull]# asadmin start-domain

Now you can access the administration console at http://localhost:4848/ by default the server does NOT bind to localhost but 0.0.0.0 so if you do not have a firewall you can access the console from outside the server too.

PS. There is a issue where the admin console might keep logging you out and asking for your credentials on every click. This is a cookie problem. Clear your browser cookies and you should be fine.

Change admin password

If you want to change the default admin user password (which is adminadmin) run the following command:

[root@heimdull]# asadmin change-admin-password --user=admin

Creating the cluster

Go to http://localhost:4848 and login. After this click on the Clusters menu option and select new, name the cluster something like my-cluster and click ok. You have now created the shell configuration for our cluster.

Now lets repeat the installation process on the cluster node. After you have installed Glassfish on the “node-agent” you are going to run a command to connect this server to the running admin-node that we installed in the initial installation steps. (Do not start the domain on the node-agent before it is connected to the admin server)

[root@heimdull]# asadmin create-node-agent --host admin-node --port 4848 --user admin my-cluster-agent-1
[root@heimdull]# asadmin start-node-agent my-cluster-agent-1
Please enter the admin user name>admin
Redirecting application output to /opt/glassfish/nodeagents/slave.local/agent/logs/server.log
Command start-node-agent executed successfull
Please enter the admin user name>admin
Please enter the admin password>password
Please enter the master password [Enter to accept the default]:>
Redirecting output to /opt/glassfish/nodeagents/slave.local/agent/logs/server.log
Redirecting application output to /opt/glassfish/nodeagents/slave.local/agent/logs/server.log
Command start-node-agent executed successfully.

You now have a running cluster of 1 node. Login to the admin console and have a look, cluster of one, nice!

Create the server instance

You do now need to create the server instance which will be running in the cluster. Click on the clusters menu and select my-cluster. Now click the Instances menu button and click new.

Name the instance instance-1 and select the newly added node agent.

You now have the following components running/installed:

– (DAS) Domain Admin Server: This is the server that will controller the “cluster”.

– (NA) Node agent: All members of the cluster needs a nodeagent that connects to the admin server. The nodeagent needs to be started on the cluster member before the admin server can do any monitoring/administration of the member.

– (SI) Server Instance: This is where all the action is, this is the actual “server” that will be added to the server and this is where you application will be running.

to_json and associations

The problem was that I wanted to get to a address that was associated with the store. I was using .to_json but this did not include the address information.

First I had var store = @store.to_json but that only gave me the address_id not the address information. After a little digging I got the syntax correct. Here is my code syntax…

This is my head tag:

<script type="text/javascript">var stores=<%=@stores.to_json(:include => {:store_address => {}})%>;</script>

Than I can use the stores variable in my google maps javascript like so:

var storePoint = new GLatLng(stores[i].store.lat, stores[i].store.lng);
var storeText = '<div id="gmap-content">'+
  '<h4>' + stores[i].store.name + '</h4>'+
  '<em>' + stores[i].store.store_address.address + '</em><br />' +
  '<em>' + stores[i].store.store_address.city + '</em><br />' +
  '<em>' + stores[i].store.store_address.phone + '</em>' +
  '</div>';
var storeMarker = new createMarker(storePoint,storeText)

PS. These are just snips and need more code to work..

Running Tomcat as a service on Linux

Running Tomcat as a service is as easy as getting jsvc to run and it’s all documented in … hmmm well anyways this is how I do it…

Compiling JSVC

[root ~]# cd $TOMCAT_HOME/bin && tar -zxf jsvc.tar.gz
[root ~]# cd jsvc-src && chmod +x configure
[root ~]# ./configure
[root ~]# make

Now there whould be a file called jsvc in $TOMCAT_HOME/bin/jsvc-src/native/ please move this file to $TOMCAT_HOME/bin/jsvc this is the Java service executable that will execute your Tomcat server. The next step is to setup your init file, lets create the file /etc/init.d/tomcat that will be used to start Tomcat with jsvc.

#!/bin/sh
##############################################################
# chkconfig: 345 98 98
### BEGIN INIT INFO
# Provides: Tomcat
# Required-Start: $local_fs $network $remote_fs
# Required-Stop: $local_fs $network $remote_fs
# Default-Start:  3 4 5
# Default-Stop: 0 1 6
# Short-Description: start and stop Tomcat
# Description: Tomcat is a J2EE Application server
### END INIT INFO

CATALINA_HOME=/usr/local/tomcat
DAEMON_HOME=/usr/local/tomcat
TOMCAT_USER=tomcat

# Make sure we load the setenv.sh file 
# since we don't use the startup.sh script
. $CATALINA_HOME/bin/setenv.sh

# for multi instances adapt these lines.
TMP_DIR=/var/tmp
PID_FILE=/var/run/jsvc.pid
CATALINA_BASE=/usr/local/tomcat

CLASSPATH=\
$JAVA_HOME/lib/tools.jar:\
$CATALINA_HOME/bin/commons-daemon.jar:\
$CATALINA_HOME/bin/bootstrap.jar

case "$1" in
  start)
    #
    # Start Tomcat
    #
    #-user $TOMCAT_USER \
    echo $"Starting Tomcat..."
    cd $CATALINA_HOME && \
    $DAEMON_HOME/bin/jsvc \
    -home $JAVA_HOME \
    -Dcatalina.home=$CATALINA_HOME \
    -Dcatalina.base=$CATALINA_BASE \
    -Djava.io.tmpdir=$TMP_DIR \
    -wait 10 \
    -pidfile $PID_FILE \
    -outfile $CATALINA_HOME/logs/catalina.out \
    -errfile '&1' \
    $CATALINA_OPTS \
    -cp $CLASSPATH \
    org.apache.catalina.startup.Bootstrap
    #
    # To get a verbose JVM
    #-verbose \
    # To get a debug of jsvc.
    #-debug \
    exit $?
  ;;

  stop)
  #
  # Stop Tomcat
  #
  echo $"Stoping Tomcat..."
    $DAEMON_HOME/bin/jsvc \
    -stop \
    -pidfile $PID_FILE \
    org.apache.catalina.startup.Bootstrap
    exit $?
  ;;

  restart)
    $0 stop
    $0 start
    exit $?
  ;;

  *)
    echo "Usage: tomcat {start|stop}"
    exit 1;;
esac

Now you can start tomcat using /etc/init.d/tomcat start. You will see TWO java processes running on your computer when Tomcat is started in this way. There is one control process and then there is the wrapper for your java process (Tomcat).