Host key verification failed

A somewhat common error with ssh which usually happens because of the following:

* a system has been updated
* if its IP has changed

A sample of the full error would be:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
eb:2b:5f:c3:5e:31:ff:c7:41:2f:c7:bd:ea:18:50:3f.
Please contact your system administrator.
Add correct host key in /Users/username/.ssh/known_hosts to get rid of this message.
Offending key in /Users/username/.ssh/known_hosts:12
RSA host key for abc-xyz.com has changed and you have requested strict checking.
Host key verification failed.


If these two possible changes have not occurred, there *may* be need for concern. However, if these sorts of changes are going on it is perfectly safe to resolve this error with one of the two following fixes.

The first is a bit easier/straight forward, but will require that you reaccept (type "yes" at first ssh login) all future server logins:

$ rm ~/.ssh/known_hosts


The second is more graceful. It will retain all your past acceptances and only require that you reaccept the current, problematic one:

1. open ~/.ssh/known_hosts in a text editor
2. Remove the line that starts with: "<server host name causing the error>,<server IP causing the error>"

Read the rest of this entry...

Service Manager

# to show what system services are running or disabled
root@server:# svcs -a

# To start, stop or restart a service
root@server:# svcadm enable ssh
root@server:# svcadm disable ssh
root@server:# svcadm refresh ssh


# to clear maintenance flag and start a service
root@server:# svcadm clear ssh
root@server:# svcadm enable ssh

# Use -t to make change non persistant
root@server:# svcadm enable -t ssh

# to switch to a milestone
root@server:# svcadm milestone svc:/milestone/single-user:default

Read the rest of this entry...

Useful commands and other programs

# See the man pages for more information
# Most can add a -v for verbose output

prtconf # General system information
prtdiag # System diag info including any current alerts
psrinfo # processor information
pagesize # show avaialble memory page sizes


# The current state of the system
vmstat
mpstat
iostat
netstat
swap

# Some other more detailed stat commands
kstat
cpustat
cputrack
busstat
trapstat

# These take a process id as last argument
##### Read man page about warnings before using these #####
pmap # Show process memory mapping
pargs # args used to start process also
pargs -e # show environment for a process
pfiles # files opn by a process - see warnings
pstack # show current stack trace - see warnings
pflags # process flags
pcred # process owner etc
pldd # dynamic libraries linked to process
psig # signal handlers and headers
pwdx # current working directory

Read the rest of this entry...

Detect new hardware on a system

root@server:# touch /reconfigure; reboot


# or from obp
root@server:# reboot -- -r

# Solaris 10
root@server:# devfsadm

Read the rest of this entry...

Checking hardware info

# Display kernel info and patch level
root@server:# uname -a

root@server:# prtdiag -v
root@server:# prtconf -v
root@server:# psrinfo -v


# Show what kernel modules are loaded
root@server:# modinfo

# Display lots of system info
root@server:# sysdef

# Display or set the library search path
root@server:# crle

Read the rest of this entry...

ufsdump and ufsrestore

# backup a filesystem
# the number refers to the backup level - i.e. 0 for full to 9 for incremental
root@server:# ufsdump 0uf /path/to/save/ufsdump-output /dev/dsk/c?t?d?s?


# make a new filesystem to restore to
root@server:# newfs /dev/rdsk/c?t?d?s?

# restore the complete backup to the CURRENT DIRECTORY
root@server:# ufsrestore rf /path/to/save/ufsdump-output
root@server:# rm restoresymtable

# unmount and check the new filesystem
root@server:# umount /mountpoint
root@server:# fsck /dev/dsk/c?t?d?s?

Read the rest of this entry...

Creating File system snapshots

# Take a snapshot of a filesystem so you can back it up. command will return the virtual device to backup
root@server:# fssnap -F ufs -o bs=/path/to/snapshot-storage-file /mountpoint
/dev/fssnap/0


#### NOTE!!! ##########################################
# If you are running ntp you may get the following error
fssnap: ioctl: error 22: Invalid argument

# If you do then stop the ntp server and try again
########################################################

# backup the filesystem using virtual device returned
root@server:# ufsdump 0uf /path/to/save/ufsdump-output /dev/fssnap/0

# remove virtual device
root@server:# fssnap -d /mountpoint

Read the rest of this entry...

Mount SVM array from a live CD

On Solaris 9 or 10

# You need to mount one of the disks read only to get a copy of the original config

root@server:# mount -o ro /dev/dsk/c?t?d?s? /a
root@server:# cp -p /a/kernel/drv/md.conf /kernel/drv/md.conf
root@server:# umount /a


P# Now load SVM:

# First remove existing module
root@server:# modinfo | grep md
154 7bb93000 2a140 85 1 md (Solaris Volume Manager base mod)
root@server:# modunload -i 154

# Now load the md module
#### For Solaris 9
root@server:# modload /kernel/drv/md

#### For Solaris 10
root@server:# modload /kernel/drv/md OR update_drv -f md

# and scan the disks
root@server:# metainit -r

# Make sure it all looks fine
root@server:# metastat -p

# If you change anything in md.conf then you MUST copy it back to the disks

Read the rest of this entry...

Delete an array in SVM

# Delete all replica databases
root@server:# metadb -d -f c?t?d?s7


# Now clear all devices
root@server:# metaclear d10
root@server:# metaclear d0

Read the rest of this entry...

Hot swap a disk from an array

he following is a procedure I used to hot swap a disk that was part of a mirror on a system.

# remove d1 and d4 devices from the arrays
root@server:# metadetach -f d0 d1
root@server:# metadetach -f d3 d4


# clear the data from the faulty disks
root@server:# metaclear d1
root@server:# metaclear d4

# clear the svm database from the faulty disk
root@server:# metadb -f -d c1t0d0s7
root@server:# format

# find where the faulty disk is
root@server:# cfgadm -alv

# remove the disk from the kernel
root@server:# cfgadm -c unconfigure c1::dsk/c1t0d0

# check the disk no longer appears
### If disk has gone then hot swap the disk ###
root@server:# format

# Tell the kernel about the device
root@server:# cfgadm -c configure c1::dsk/c1t0d0

# copy the vtoc from existing disk to new disk
root@server:# prtvtoc /dev/rdsk/c1t1d0s0 > /vtoc.c1t1d0s0
root@server:# fmthard -s /vtoc.c1t1d0s0 /dev/rdsk/c1t0d0s0

# Initialize and add the new disk to each array
root@server:# metainit d1 1 1 c1t0d0s1
root@server:# metattach d0 d1
root@server:# metainit d4 1 1 c1t0d0s0
root@server:# metattach d3 d4

# create the svm databases
root@server:# metadb -a -c 3 c1t0d0s7

# make the new disk bootable
root@server:# installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c1t0d0s0

Read the rest of this entry...

Replace a faulty drive from an array

Array was d0 with d10 and d20 as submirrors

# Detach the faulty disk
root@server:# metadetach -f d0 d10

# and remove the disk from the array
root@server:# metaclear d10


# delete the SDR from the disk
root@server:# metadb -f -d c?t?d?s?

# If hot swap then replace the disk

# Copy the vtoc from the good disk to a file
root@server:# prtvtoc /dev/rdsk/c?t?d?s? > vtoc.c?t?d?s?

# and copy to the new disk
root@server:# fmthard -s vtoc.c?t?d?s? /dev/rdsk/c?t?d?s?

# Add the new disk to the array
root@server:# metainit d10 1 1 c?t?d?s?

# and atach to the array. Will sync to existing drive
root@server:# metattach d0 d10

# add an SDR to the new disk
root@server:# metadb -a -c 1 c?t?d?s?

Read the rest of this entry...

Related Posts on Blogger

The related posts widget is easily available in wordpress, all you have to do is install the widget. Well that's ok "if you're using Wordpress" but how about Blogger user. Here's a script we can use to have Related Posts on Blogger.

1. Go to your template editor, Under your blogge /blogspot dashboard.
Click on the donwnload full template link, save your downloaded template as a back up

2. Find this code below:

<data:post.body/>

3. Copy this code below:

<b:if cond='data:blog.pageType == &quot;item&quot;'>
<div class='similiar'>

<div class='widget-content'>
<h3>Related Posts:</h3>
<div id='data2007'/><br/><br/>

<script type='text/javascript'>

var homeUrl3 = &quot;<data:blog.homepageUrl/>&quot;;
var maxNumberOfPostsPerLabel = 4;
var maxNumberOfLabels = 10;

maxNumberOfPostsPerLabel = 100;
maxNumberOfLabels = 3;


function listEntries10(json) {
var ul = document.createElement(&#39;ul&#39;);
var maxPosts = (json.feed.entry.length &lt;= maxNumberOfPostsPerLabel) ?
json.feed.entry.length : maxNumberOfPostsPerLabel;
for (var i = 0; i &lt; maxPosts; i++) {
var entry = json.feed.entry[i];
var alturl;

for (var k = 0; k &lt; entry.link.length; k++) {
if (entry.link[k].rel == &#39;alternate&#39;) {
alturl = entry.link[k].href;
break;
}
}
var li = document.createElement(&#39;li&#39;);
var a = document.createElement(&#39;a&#39;);
a.href = alturl;

if(a.href!=location.href) {
var txt = document.createTextNode(entry.title.$t);
a.appendChild(txt);
li.appendChild(a);
ul.appendChild(li);
}
}
for (var l = 0; l &lt; json.feed.link.length; l++) {
if (json.feed.link[l].rel == &#39;alternate&#39;) {
var raw = json.feed.link[l].href;
var label = raw.substr(homeUrl3.length+13);
var k;
for (k=0; k&lt;20; k++) label = label.replace(&quot;%20&quot;, &quot; &quot;);
var txt = document.createTextNode(label);
var h = document.createElement(&#39;b&#39;);
h.appendChild(txt);
var div1 = document.createElement(&#39;div&#39;);
div1.appendChild(h);
div1.appendChild(ul);
document.getElementById(&#39;data2007&#39;).appendChild(div1);
}
}
}
function search10(query, label) {

var script = document.createElement(&#39;script&#39;);
script.setAttribute(&#39;src&#39;, query + &#39;feeds/posts/default/-/&#39;
+ label +
&#39;?alt=json-in-script&amp;callback=listEntries10&#39;);
script.setAttribute(&#39;type&#39;, &#39;text/javascript&#39;);
document.documentElement.firstChild.appendChild(script);
}

var labelArray = new Array();
var numLabel = 0;

<b:loop values='data:posts' var='post'>
<b:loop values='data:post.labels' var='label'>
textLabel = &quot;<data:label.name/>&quot;;

var test = 0;
for (var i = 0; i &lt; labelArray.length; i++)
if (labelArray[i] == textLabel) test = 1;
if (test == 0) {
labelArray.push(textLabel);
var maxLabels = (labelArray.length &lt;= maxNumberOfLabels) ?
labelArray.length : maxNumberOfLabels;
if (numLabel &lt; maxLabels) {
search10(homeUrl3, textLabel);
numLabel++;
}
}
</b:loop>
</b:loop>
</script>
</div>

</div>
</b:if>

3. Paste right after the code in step 2 (<data:post.body/>)

<p><data:post.body/></p> ADD_the_related_posts_script_here

4. Save your template. View your blog, and you're finish.

A similar article can be found at jackbook.com do try to read it as well.

Read the rest of this entry...

Doing an SVM Raid on existing single partition

# Make sure both disks have the same partition layout and create slices for the replica database (2 cylinders)
# Check the disks and the layout
root@server:# format


# You can copy the vtoc from one disk to another with this.
# You can also copy the vtoc to/from a file
root@server:# prtvtoc /dev/rdsk/c?t?d?s? | fmthard -s - /dev/rdsk/c?t?d?s?

# For x86 machines set the active partition for the disks
root@server:# fdisk -b /usr/lib/fs/ufs/mboot /dev/rdsk/c?t?d?p?

# If making root partition raid then make second disk bootable
# For x86 machines
root@server:# /sbin/installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c?t?d?s?

# And for Sparc machines
root@server:# installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c?t?d?s?


# Add the State Database Replicas to a slice (normally s7) - may need -f to force if partition mounted
root@server:# metadb -a -f c?t?d?s?
# Need more than one copy. Spread accross disks/controllers etc - see docs for details
root@server:# metadb -a -f c?+1t?d?s?

# We will use d0 with d10 and d20 as the two mirrors


# Initialise the first mirror in the array. -f to force and "1 1" for one stripe and one slice
root@server:# metainit -f d10 1 1 c?t?d?s?

# Initialise the second mirror in the array
root@server:# metainit d20 1 1 c?t?d?s?

# Add the first mirror to the raid array
root@server:# metainit d0 -m d10

# If doing the root partition this will update /etc/vfstab to point to new root partition location. If doing non-root partitions then vi the file
root@server:# metaroot d0

# Reboot so system uses half of the new raid 1 array
root@server:# reboot

# Add the second mirror to the raid array
root@server:# metattach d0 d20

# To check on the status of the array and make sure it is syncing
root@server:# metastat

# Misc commands
# To delete an SDR
root@server:# metadb -d -f c?t?d?s?

# Detach a mirror from an array
root@server:# metadetach d0 d10

# Take mirror off line
root@server:# metaoffline d0 d20

# Put mirror back in array and resync
root@server:# metaonline d0 d20

# Create a UFS file system on the new array
root@server:# newfs /dev/md/dsk/d?

# Backup your metadb information
root@server:# metastat -p > /etc/lvm/md.tab

Read the rest of this entry...

Creating a raid mirror on two empty partitions

# create one side of the mirror on the partition
root@server:# metainit d11 1 1 c0t1d0s0


# and the other side of the mirror
root@server:# metainit d12 1 1 c0t2d0s0

# now create a one sided mirror
root@server:# metainit d10 -m d11

# attach the second side to the existing one sided mirror
root@server:# metattach d10 d12

Read the rest of this entry...

ZFS again

I've been receiving pm's for some more zfs commands from people who have read my previous posts regarding zfs, namely Solaris ZFS CBT Notes, ACLs for serving websites from a ZFS filesystem (Sol) and Deleting ZFS slices from a disk.

So here's some more commands:

# Some reminders on command syntax
root@server:# zpool create stuff c0d0s0
root@server:# zpool create stuff mirror c0d0s3 c1d0s0
root@server:# zfs create stuff/www
root@server:# zfs set mountpoint=/var/www stuff/www
root@server:# zfs create stuff/home
root@server:# zfs set mountpoint=/export/home stuff/home
root@server:# zfs create stuff/home/peter
root@server:# zfs create stuff/home/paul
root@server:# zfs create stuff/home/mary
root@server:# zfs set compression=on stuff/home
root@server:# zfs set quota=5g stuff/home/paul
root@server:# zfs set reservation=10g stuff/home/mary
root@server:# zfs set sharenfs=rw stuff/home
root@server:# zpool scrub stuff
root@server:# zpool add stuff mirror c3d0s4 c4d1s2

# to set the filesystem block size to 8k
root@server:# zfs set recordsize=8k stuff/home

Read the rest of this entry...

SVM Soft Partitions

Got hold of my old copy of an old tutorial for SVM regarding soft partitions, I was still able to see where I got it originally but I think the site is no longer there.

originally posted at: http://www2.primushost.com/~griff/soft-partitions.html

Solstice DiskSuite / Solaris Volume Manager

Soft Partitioning


A Primer for Understanding Soft Partitioning,
a new feature in Solstice DiskSuite (Solaris Volume Manager)

The intent of this document is to describe Soft Partitioning within Solstice DiskSuite (soon-to-be-renamed Solaris Volume Manager), and offer a short primer/tutorial on how to create, use, and delete them.

Until now, Solaris, without any volume management software, has only ever allowed a fixed number of partitions on a physical disk (seven (7) on SPARC platforms). With the increase in capacity of disks, this limitation has become a severe restriction.

SDS/SVM uses these slices for its metadevices (sub-mirrors, trans, stripes, and RAID5) and hence is faced with the same limitation, whereas Veritas Volume Manager (VxVM) allows for the logical partitioning of disks into a virtually unlimited number of subdisks.

Soft Partitioning allows for a disk to be subdivided into many partitions which are controlled and maintained by software, thereby removing the limitation of the number of partitions on a disk. A soft partition is made up of one or more "extents". An extent describes the parts of the physical disk that make up the soft partition. While the maximum number of extents per soft partition is 2147483647, the majority of soft partitions will use only one (1) extent.

What is new?

Soft Partitioning was not in the original Solstice DiskSuite 4.2.1 Release, which coincided with the release of Solaris 8. However, the soft partitioning functionality was released in patch 108693-06 for SDS 4.2.1.

When Solaris 9 gets released, the "Solstice DiskSuite" name will change to "Solaris Volume Manager" ("SVM") and it will be bundled in with Solaris 9. Soft Partitioning will, of course, be part of the base functionality of that release.

Soft Partitions are implemented by new kernel driver: md_sp.

# modinfo | grep md_sp
228 78328000 4743 - 1 md_sp (Meta disk soft partition module)

There are new options to the metainit command:

metainit softpart -p [-e] component size
metainit softpart -p component -o offset -b size

The metattach command has been modified to allow for growing of soft partitions:

metattach softpart size

There is a new command... metarecover:

metarecover [-n] [-v] component -p [-d|-m]

NOTE: the -p option means that the command refers to soft partitions.

Creating Soft Partitions
There are three methods to create a soft partition using the metainit command:

1. Specifying an unused disk and size (with the -e option). For example:

# metainit d0 -p -e c1t0d0 200m

The -e option requires that the name of the disk supplied be in the form c#t#d#.

The last parameter (200m) specifies the initial size of the soft partition. The sizes can be specified in blocks, kilobytes, megabytes, gigabytes, and terabytes.

The -e option causes the disk to be repartitioned such that slice 7 has enough space to hold a replica (although no replica is actually created on this disk) and slice 0 contains the rest of the space. Slice 2 is removed from the disk. The soft partition that is being created is put into slice 0. Further soft partitions can be created on slice 0 by the next method of creating a soft partition.

After this command is run, the layout of the disk would like similar to this example:

Part Tag Flag Cylinders Size Blocks
0 unassigned wm 5 - 2035 999.63MB (2031/0/0) 2047248
1 unassigned wm 0 0 (0/0/0) 0
2 unassigned wm 0 0 (0/0/0) 0
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 unassigned wm 0 0 (0/0/0) 0
7 unassigned wu 0 - 4 2.46MB (5/0/0) 5040

This command (with the -e) can only be run on an empty disk (one that is not used in any other metadevice). If another metadevice or replica already exists on this disk, one of the following messages will be printed, and no soft partition will be created.

metainit: hostname: c#t#d#s0: has appeared more than once in the specification of d#

or

metainit: hostname: c#t#d#s#: has a metadevice database replica

2. Specifying an existing slice name and size (without the -e option). This will be the most common method of creation. For example:

# metainit d1 -p c1t0d0s0 1g

This will create a soft partition on the specified slice. No repartitioning of the disk is done. Provided there is space on the slice, additional soft partitions could be created as required. The device name must include the slice number (c#t#d#s#).

If another soft partition already exists in this slice, this one will be created immediately after the existing one. Therefore, no overlap of soft partitions can occur by accident.

3. Specifying an existing slice and absolute offset and size values. For example:

# metainit d2 -p c1t0d0s0 -o 2048 -b 1024

The -o parameter signifies the offset into the slice, and the -b parameter is the size for the soft partition. All numbers are in blocks (a block is 512 bytes). The metainit command ensures that extents and soft partitions do not overlap. For example, the following is an attempt to create overlapping soft partitions.

# metainit d1 -p c1t0d0s0 -o 1 -b 2024
d1: Soft Partition is setup
# metainit d2 -p c1t0d0s0 -o 2000 -b 2024
metainit: hostname: d2: overlapping extents specified

An offset of 0 is not valid, as the first block on a slice containing a soft partition contains the initial extent header. Each extent header consumes 1 block of disk and each soft partition will have an extent header placed at the end of each extent. Extent headers are explained in more detail in the next section.

NOTE: This method is not documented in the man page for metainit and is not recommended for manual use. It is here because a subsequent metastat -p command will output information in this format.


Extent Headers

Whenever a soft partiton is created in a disk slice, an "extent header" is written to disk. Internally to Sun, these are sometimes referred to as "watermarks".

An extent header is a consistency record and contains such information as the metadevice (soft partition) name, it's status, it's size, and a checksum. Each extent header 1 block (512 bytes) in size.

The following diagram shows an example 100MB slice (c1t0d0s0) and the extent headers (watermarks) that have been created on it. The command to make the soft partition shown was

# metainit d1 -p c1t0d0s0 20m

->>

There is always an extent header on the first and last blocks in the slice. Note that the 80MB of space left over from the creation of the soft partition can be used to make one or more additional soft partitions. Each additional soft partition will create an additional extent header to be created as well.

Mirroring Soft Partitions
Once you have created soft partitions, what can you do with them? Well, one thing to do is to create mirrors out of them. Unfortunately, even though a soft partition is a metadevice, they cannot serve directly as a submirror. For example:

# metainit d10 -p c1t11d0s4 100m
d10: Soft Partition is setup
# metainit d20 -m d10
metainit: hostname: d10: invalid unit

Instead, you must first take the soft partition and create a simple concat/stripe out of it. For example:

# metainit d10 -p c1t0d0s0 100m
d10: Soft Partition is setup
# metainit d20 1 1 d10
d20: Concat/Stripe is setup
# metainit d30 -m d20
d30: Mirror is setup

# metainit d11 -p c2t0d0s0 100m
d11: Soft Partition is setup
# metainit d21 1 1 d11
d21: Concat/Stripe is setup
# metattach d30 d21
d30: submirror d21 is attached

Once done, the resulting metastat output of the mirror will look like this:

# metastat d30

d30: Mirror
Submirror 0: d20
State: Okay
Submirror 1: d21
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 204624 blocks

d20: Submirror of d30
State: Okay
Size: 204624 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
d10 0 No Okay

d10: Soft Partition
Component: c1t0d0s0
State: Okay
Size: 204800 blocks
Extent Start Block Block count
0 1 204800

d21: Submirror of d30
State: Okay
Size: 204624 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
d11 0 No Okay

d11: Soft Partition
Component: c2t0d0s0
State: Okay
Size: 204800 blocks
Extent Start Block Block count
0 1 204800


Combining Soft Partitions Together into a RAID5 Device

RAID5 devices can be made up of soft partitions directly. This example shows 4 soft partitions (from 4 separate slices) striped together to make a RAID5 device:

# metainit d1 -p c1t0d0s0 10m
d1: Soft Partition is setup
# metainit d2 -p c2t0d0s0 10m
d2: Soft Partition is setup
# metainit d3 -p c3t0d0s0 10m
d3: Soft Partition is setup
# metainit d4 -p c4t0d0s0 10m
d4: Soft Partition is setup
# metainit d10 -r d1 d2 d3 d4
d10: RAID is setup

Once done, the resulting metastat output of the RAID5 device will look like this:

# metastat d10

d10: RAID
State: Okay
Interlace: 32 blocks
Size: 59472 blocks
Original device:
Size: 60384 blocks
Device Start Block Dbase State Hot Spare
d1 330 No Okay
d2 330 No Okay
d3 330 No Okay
d4 330 No Okay

d1: Soft Partition
Component: c1t0d2s0
State: Okay
Size: 20480 blocks
Extent Start Block Block count
0 1 20480

d2: Soft Partition
Component: c1t0d4s0
State: Okay
Size: 20480 blocks
Extent Start Block Block count
0 1 20480

d3: Soft Partition
Component: c1t1d1s0
State: Okay
Size: 20480 blocks
Extent Start Block Block count
0 1 20480

d4: Soft Partition
Component: c1t1d3s0
State: Okay
Size: 20480 blocks
Extent Start Block Block count
0 1 20480


Using Soft Partitions for MetaTrans (UFS Logging) Devices
MetaTrans devices (UFS logging) can be built on top of soft partitions. Soft partitions can be used for the master device, the logging device, or both. In the following example, soft partitions are used for both the master and the logging device:

# metainit d1 -p c1t0d0s0 500m
d1: Soft Partition is setup
# metainit d2 -p c2t0d0s0 50m
d2: Soft Partition is setup
# metainit d10 -t d1 d2
d1: Trans is setup

Once done, the resulting metastat output of the metatrans device will look like this:

# metastat d10
d10: Trans
State: Okay
Size: 1024000 blocks
Master Device: d1
Logging Device: d2

d1: Soft Partition
Component: c1t1d3s0
State: Okay
Size: 1024000 blocks
Extent Start Block Block count
0 1 1024000

d2: Logging device for d10
State: Okay
Size: 102142 blocks

d2: Soft Partition
Component: c1t1d1s0
State: Okay
Size: 102400 blocks
Extent Start Block Block count
0 1 102400


Layering

Most of the time, soft partitions are made on a disk slice. However, there are certain situations where it can be beneficial to make a soft partition on top of an existing metadevice. This is referred to as layering.

For example, say you have a 90GB RAID5 device made up of 6 18GB disks. You can then take that 90GB device and "split it up" into many soft partitions. These many soft partitions then can be accessed as separate simple metadevices, although the data in them is protected by the RAID5 parity in the underlying device.

Soft partitions can be layered only on top of concat/stripes, mirrors, and RAID5 devices. Soft partitions cannot be layered on top of a metatrans device or directly on top of another soft partition.

Here is an example of layering soft partitions on top of an existing RAID5 metadevice. First, we create the RAID5 device, then soft partition that device into 3 100MB partitions (obviously, we could create more than just 3 soft partitions).

# metainit d0 -r c1t0d2s0 c1t0d4s0 c1t1d1s0 c1t1d3s0
d0: RAID is setup

# metainit d1 -p d0 100m
d1: Soft Partition is setup
# metainit d2 -p d0 100m
d2: Soft Partition is setup
# metainit d3 -p d0 100m
d3: Soft Partition is setup

Each of the resulting soft partitions (d1, d2, and d3) can be accessed individually (i.e., newfs and mount).

Soft partitions can be built on top of an existing mirror device as well, just like we did above on the RAID5 device. In the following example, the mirror device (d0) is "carved up" into 3 smaller soft partitions.

# metainit d10 1 1 c1t0d2s0
d10: Concat/Stripe is setup
# metainit d20 1 1 c2t0d0s0
d20: Concat/Stripe is setup
# metainit d0 -m d10 d20
d0: Mirror is setup

# metainit d1 -p d0 100m
d1: Soft Partition is setup
# metainit d2 -p d0 100m
d2: Soft Partition is setup
# metainit d3 -p d0 100m
d3: Soft Partition is setup

Soft partitions are not allowed to be parented by other soft partitions directly. For example:

# metainit d1 -p c1t0d0s0 100m
d1: Soft Partition is setup
# metainit d2 -p d1 10m
metainit: hostname: d1: invalid unit

Soft partitions also cannot be built on top of trans (UFS logging) devices. For example:

# metainit d1 -t d10 d20
d1: Trans is setup
# metainit d2 -p d1 100m
metainit: hostname: d1: invalid unit


Growing Soft Partitions

A soft partition can be grown by the use of the metattach command. There is no mechanism to shrink a soft partition.

# metattach d0 10m
d0: Soft Partition has been grown

When additional space is added to an existing soft partition, the additional space is taken from any available space on the same device and might not be contiguous with the existing soft partition. Growing soft partitions must be done with free space in the same device as the current soft partition.

The following example shows how growing a soft partition will increase the size of the current extent:

# metainit d1 -p c1t0d2s0 100m
d1: Soft Partition is setup
# metastat d1
d1: Soft Partition
Component: c1t0d2s0
State: Okay
Size: 204800 blocks
Extent Start Block Block count
0 1 204800

# metattach d1 50m
d1: Soft Partition has been grown
# metastat d1
d1: Soft Partition
Component: c1t0d2s0
State: Okay
Size: 307200 blocks
Extent Start Block Block count
0 1 307200

Note how after the metattach is run, there is still only one extent, but the (block count) has grown from 204800 (100MB) to 307200 (150MB).

In the following example, the extent cannot be grown, as it was above, because another soft partition is "in the way". Therefore, a second extent is created in the same slice.

# metainit d1 -p c1t0d2s0 100m
d1: Soft Partition is setup
# metainit d2 -p c1t0d2s0 10m
d2: Soft Partition is setup
# metastat
d1: Soft Partition
Component: c1t0d2s0
State: Okay
Size: 204800 blocks
Extent Start Block Block count
0 1 204800

d2: Soft Partition
Component: c1t0d2s0
State: Okay
Size: 20480 blocks
Extent Start Block Block count
0 204802 20480

# metattach d1 50m
d1: Soft Partition has been grown
# metastat
d1: Soft Partition
Component: c1t0d2s0
State: Okay
Size: 307200 blocks
Extent Start Block Block count
0 1 204800
1 225283 102400

d2: Soft Partition
Component: c1t0d2s0
State: Okay
Size: 20480 blocks
Extent Start Block Block count
0 204802 20480

Note how d1 now has two non-contiguous extents that together make up the 307200 (150MB) blocks.

NOTE: Growing the metadevice does not modify the data or the filesystem inside the metadevice. If the metadevice contains a filesystem, you must use the appropriate command(s) to grow that filesystem after the metadevice has been grown.

Deleting Soft Partitions
This is achieved by using the metaclear command in the normal way:

# metaclear d0
d0: Soft Partition is cleared

If other metadevices are using the soft partition, the metaclear will error with:

metaclear: hostname: d0: metadevice in use


Using Soft Partitions with Disksets

There are no differences with soft partitioning in a diskset, other than having to specify the -s option on the commandline to specify the diskset name.

The only potential problem occurs when dealing with did disk devices that are in a SunCluster configuration. Unfortunately, the naming convention of the did devices is similar to that of SDS/SVM in that the disks are referred to as d#. This means that SDS/SVM could confuse a did disk with a metadevice when creating a soft partition.

The simple workaround to this problem is to use the full path to the did device on the metainint commandline in order to prevent any confusion.

For example, the following command to create a 1GB soft partition on /dev/did/rdsk/d7s0 would be invalid:

# metainit -s set2 d0 -p d7s0 1g

Instead, the correct command to run would be:

# metainit -s set2 d0 -p /dev/did/rdsk/d7s0 1g


How to list the soft partitions in a given slice

The metarecover command, with the -n and -v options, will display information about the soft partitons existing in a given slice.

The metarecover command actually scans the given slice for extent headers and prints the information that it finds about those headers.

In each slice/device, there are also 2 additional extent headers; one which preceeds the free space in the slice, and the one on the last block of the slice. These are printed as well. This is an easy way to determine how much free space is available in a slice for additional soft partitions.

# metarecover -v -n /dev/rdsk/c1t0d0s0 -p
Verifying on-disk structures on c1t0d0s0.
The following extent headers were found on c1t0d0s0.
Name Seq# Type Offset Length
d0 0 ALLOC 0 20481
d1 0 ALLOC 20481 40961
NONE 0 END 17674901 1
NONE 0 FREE 61442 17613459
Found 2 soft partition(s) on c1t0d0s0.

In the above example, there were 2 soft partitions (d0 and d1) found on c1t0d0s0, as well as 17613458 blocks (approx 8.4GB) of unallocated free space.

IMPORTANT NOTE: The information printed by this command is relative to the extent header, not the soft partition itself. Therefore, the 'offset' is the starting location of the extent header, not the extent itself. Also, the 'length' given is the length of the extent plus the header. Therefore, in the example above, there are only 17613458 free blocks, not 17613459 blocks.

Because soft partitions can be layered above metadevices like mirrors or RAID5 devices (see layering, above), this command can also be run on them to determine the locations and sizes of the extent headers. In the example below, d0 is a RAID5 metadevice which has 4 soft partitions in it. There is no free space left in this device.

# metarecover -v -n d0 -p
Verifying on-disk structures on d0.
The following extent headers were found on d0.
Name Seq# Type Offset Length
d1 0 ALLOC 0 204801
d2 0 ALLOC 204801 204801
d3 0 ALLOC 409602 204801
d99 0 ALLOC 614403 7573580
NONE 0 END 8187983 1
Found 4 soft partition(s) on d0.


Fragmentation

Fragmentation of free space will occur on a slice when there has been activity in creating, deleting, and possibly growing soft partitions. At this time, there is no method to defragment a disk.

For example, the following sequence of commands can result in some amount of fragmentation. First, create 2 10MB soft partitions on a slice.

# metainit d1 -p c1t0d0s0 10m
d1: Soft Partition is setup
# metainit d2 -p c1t0d0s0 10m
d2: Soft Partition is setup

Then, remove the first 10MB soft partition and then create a 20MB soft partition.

# metaclear d1
d1: Soft Partition is cleared
# metainit d3 -p c1t0d0s0 20m
d3: Soft Partition is setup

When the d3 metadevice was created, the 10MB of free space at the beginning of the slice is not used, because there is a contiguous 20MB space available further out that can be used instead. Therefore, the 10MB of free space is skipped over in favor of the first 20MB of contiguous space. The metarecover command will show the fragmentation (multiple free spaces):

# metarecover -v -n c1t0d0s0 -p
Verifying on-disk structures on c1t0d0s0.
The following extent headers were found on c1t0d0s0.
Name Seq# Type Offset Length
d2 0 ALLOC 20481 20481
d3 0 ALLOC 40962 40961
NONE 0 END 2047247 1
NONE 0 FREE 81923 1965324
NONE 0 FREE 0 20481
Found 2 soft partition(s) on c1t0d0s0.


Recovering Soft Partitions
The 'metarecover' command is run when something has gone wrong. It should not be run except to recover from a catastrophic problem. There are two main functions that this command does. It can

1. scan through the given slice and recreate the soft partitions that it finds there. this is good when moving a disk with soft partitions to a new machine. The option to use on the metarecover command is -d.

2. reads through the current replica and creates the soft partitions on the given slice. This is good to run after a disk fails and gets replaced with a new one. The option to use on the metarecover command is -m.

Recreating Information in the Replica from the Extent Headers
Here is a very simple example showing a disk which had soft partitions created on it (in slice 0) on another host, which is being moved to a new machine. We wish to extract the soft partitions on this new machine. Currently, there are no metadevices created.

# metastat

This command scans the given slice (in this case, "c0t0d0s0") and, for each soft partition it finds in that slice, it puts an entry into the current replica. The data on the disk is not modified, and nothing on the slice specified is modified. All that happens is that the extent headers are read and information is written to the replica.

# metarecover c0t0d0s0 -p -d
The following soft partitions were found and will be added to
your metadevice configuration.
Name Size No. of Extents
d1 61440 1
d2 20480 1
WARNING: You are about to add one or more soft partition
metadevices to your metadevice configuration. If there
appears to be an error in the soft partition(s) displayed
above, do NOT proceed with this recovery operation.

Are you sure you want to do this (yes/no)? yes

c0t0d0s0: Soft Partitions recovered from device.

Now, we can see the soft partition metadevices have been created for us:

# metastat
d1: Soft Partition
Component: c0t0d0s0
State: Okay
Size: 61440 blocks
Extent Start Block Block count
0 120836 61440

d2: Soft Partition
Component: c0t0d0s0
State: Okay
Size: 20480 blocks
Extent Start Block Block count
0 20482 20480

Recreating Soft Partitions from Information in the Replica
This example essentially does the opposite of example 1. In this case, the actual extent headers on the disk have been lost, either because something wrote over them, or because the disk hosting the soft partitions had to be replaced with new disk drive. Although the replica shows the soft partitions to be "Okay":

# metastat
d1: Soft Partition
Component: c0t0d0s0
State: Okay
Size: 61440 blocks
Extent Start Block Block count
0 120836 61440

d2: Soft Partition
Component: c0t0d0s0
State: Okay
Size: 20480 blocks
Extent Start Block Block count
0 20482 20480

there are no extent headers on the disk, so I/O to the disk will error out.

# dd if=/dev/zero of=/dev/md/rdsk/d2
dd: /dev/md/rdsk/d2: open: I/O error

To check the disk to see if any extent headers exist on the disk, you can run the command

# metarecover -n c0t0d0s0 -p
found incorrect magic number 0, expected 20000127.
No extent headers found on c0t0d0s0.
c0t0d0s0: On-disk structures invalid or no soft partitions found.
metarecover: hostname: d0: bad magic number in extent header

The above command confirms that there are no extent headers on the disk. To have the extent headers written out to the disk, according to the information currently in the replica, run the command

# metarecover c0t0d0s0 -p -m
c0t0d0s0: Soft Partition metadb configuration is valid

WARNING: You are about to overwrite portions of c0t0d0s0
with soft partition metadata. The extent headers will be
written to match the existing metadb configuration. If
the device was not previously setup with this
configuration, data loss may result.

Are you sure you want to do this (yes/no)? yes

c0t0d0s0: Soft Partitions recovered from metadb

Now, the extent headers have been written to the disk, so I/O will work correctly now. Running the verify command again, we see

# metarecover -n c0t0d0s0 -p
c0t0d0s0: Soft Partition metadb configuration is valid
c0t0d0s0: Soft Partition metadb matches extent header configuration

Read the rest of this entry...

ILOM infos

Here are some ILOM informations you may use. ILOM is the system management firmware you can use to monitor, manage, and configure a variety of Sun server platforms. If you are a UNIX administrator you would normally be familiar to it as a way to console servers.

Sun ILOM on v20z

To setup the ilom you can select Use Defaults from front panel and then add an admin user with -
ssh setup@ipaddress

To enable serial over lan -
ssh admin@ipaddress platform set console -s sp -e -S 9600

To disable -
ssh admin@ipaddress platform set console -s platform

Connect to the ilom with ssh then to start the console -
platform console

To exit the console and return to the ilom
CTRL-E c .
or you can disconnect the ssh session with
~.


To power on/off the system -
platform set power state on
platform set power state off

To display the mac address -
platform get mac


Sun ILOM on x4100

To set static ip address -
log in
cd /SP/network
set pendingipaddrss=x.x.x.x
set pendingnetmask=255.255.255.x
set pendinggateway=y.y.y.y
set commitpending=true

show /SP/network

To get to the console -
start /SP/console

To exit from console -
ESC-SHIFT-9

To power on/off and reset the system -
start /SYS
stop /SYS
reset /SYS

show /SYS

To reboot the ilom -
reset /SP


Sun x4150 ILOM

Show the IP address info for the ilom
show /SP/network

Set the IP address for the ilom
set /SP/network IPSource=static
set /SP/network IPAddress=xxx.xxx.xxx.xxx
set /SP/network Netmask=xxx.xxx.xxx.xxx
set /SP/network Gateway=xxx.xxx.xxx.xxx

Power on, off and reset the host
start /SYS
stop /SYS
reset /SYS

Power off the host gracefully
set /SYS/CtrlInfo PowerCtrl=graceful_off

Reboot to the BIOS
set /SYS/CtrlInfo PowerCtrl=BIOSSetup

Reboot to the diagnosis program
set /SYS/CtrlInfo PowerCtrl=PCCheck_enable

Start the console
start /SP/AgentInfo/Console

When started use ESC-SHIFT-9 to return to the SP (may need to try a few times)

Stop the console
stop /SP/AgentInfo/Console

Display information on serial port settings (external and host)
show /SP/serial/external
show /SP/serial/host

Set the serial port settings for terminal access
set /SP/serial/host pendingspeed=57600 commitpending=true


Sun T5120 ILOM

# Show the IP address info for the ilom
show /SP/network

# Set the IP address for the ilom
-> set /SP/network state=enabled
-> set /SP/network pendingipdiscovery=static
-> set /SP/network pendingipaddress=10.8.31.163
-> set /SP/network pendingipnetmask=255.255.252.0
-> set /SP/network pendingipgateway=10.8.31.248
-> set /SP/network commitpending=true

-> set/SP/pendingipdiscovery=dhcp
-> set /SP/network commitpending=true

# Power on, off and reset the host
start /SYS
stop /SYS
reset /SYS

# Start the console
start /SP/AgentInfo/Console

When started use #. to return to the SP (may need to try a few times)

# Create a user that has the ALOM interface - Needed if you want to upgrade the system firmware
create /SP/users/admin role=Administrator cli_mode=alom

Read the rest of this entry...

Expanding soft partitions

In the previous post create a soft partition with SVM we saw how to create a soft partition using SVM. Now this is how it can be growed or expanded.

Expanding a soft partition is a two stage process. Firstly, the partition must be increased in size and then the filesystem on the partition must be extended to use the available space.

To expand a soft partition use the following command:

metattach

For example:

metattach d0 10g

would give soft partition d0 an additional 10Gb of space.

To extend the filesystem, the growfs command is used. The syntax is:

growfs -M

For example:

growfs -M /disk/ptn001 /dev/md/rdsk/d0

The new space allocated to the partition should now be usable.

Note that growfs requires that you give it the RAW device name. Not doing so will result in an obscure error message.

Read the rest of this entry...

create a soft partition with SVM

Here we will show how to create a soft partition using Solaris volume manager or SVM on a local partition. Once created, the soft partition can be treated as an ordinary partition, allowing the creation of a filesystem. These kinds of partitions are referred to using the meta-device (/dev/md/) hierarchy. One benefit of soft partitions is that they can be expanded (with metattach & growfs).

There are two phases to creating a soft partition:

* defining a soft partition meta-device using the appropriate Solaris tools.
* creating the file system on the newly-defined soft partition.

Once you have decided on the size of the soft partition, and which physical device it should sit on, you need to create the soft partition with the metainit command.

For example, to create a 30Gb soft partition using meta-device "d3" on physical device c4t8d0s0, first define the meta-device:

# metainit d3 -p c4t8d0s0 30g

(the "-p" option defines the named meta-device "d3" as a 'soft partition' type - for other possible types, see the md.cf(4) man page). This will produce the output:

d3: Soft Partition is setup

- which defines a metadevice "d3" as a soft partition ("-p"), on disk slice c4t8d0s0, starting at offset 1 and extending for 62914560 blocks.

There are two things to note about this process. Firstly, it is possible (and indeed normal) to have more than one soft partition on a given device. Metainit will take care of setting up the offset and extent for each device.

Secondly, for the sake of disaster recovery, it would be very helpful if the names of metadevices, ie "d3" could be unique across a site. This is not the case at time of writing (18/11/04) but will hopefully become so shortly. The device name should be chosen with this in mind. To see all the metadevices defined on a server, use the metastat -p command.

Once this is done, the normal filesystem creation and checking commands can be used. For example:


# newfs /dev/md/dsk/d3

In order to mount this new filesystem, a partition name must be reserved in the "partitions" rfe map - this should be a unique name across all Informatics servers. Take the next free ptn number, and create an entry in the "partitions" rfe map - for example:

ptn028 pegasus.inf.ed.ac.uk /disk/ptn028

Next, add an entry in /etc/vfstab to allow the filesystem to be mounted. For example:


/dev/md/dsk/d3 /dev/md/rdsk/d3 /disk/ptn028 ufs 3 yes quota

NoteIf you're creating a partition that is not for home directories, eg a shared group partition, then omit the quota mount option.

Once this is done, the mount-point should be created and the filesystem mounted. You should then see something like:

# df -hF ufs
Filesystem size used avail capacity Mounted on
...
/dev/md/dsk/d3 30G 9K 29G 1% /disk/ptn028
#

To make the partition available via nfs, edit the server's LCFG profile and add the following lines:

!nfs.exports mADD()
nfs.fs_ /disk/
nfs.options_ rw=.inf.ed.ac.uk:.dcs.ed.ac.uk:.cogsci.ed.ac.uk:.dai.ed.ac.uk

This will add the partition to the /etc/dfs/dfstab file on the server. Once the dfstab file has been updated, run exportfs -a (make sure you've mounted the partition first) to make the partition generally available.

Read the rest of this entry...

Netbook Article

Was reading this article regarding the future of netbook at CNN by Wes Finley-Price. I myself don't own a "netbook" but I do have a 9 year old Fujitsu Lifebook (it was a 2000 model) touchscreen that's about as thin as the netbooks nowadays but around twice the size. I t still is enough for basic computing and use it more than my 17 inch laptops. I found this CNN article quite good and thought of sharing it here. So here it it:

I don’t own a netbook, but I do own a 5-year-old Dell Inspiron laptop. I like to think of my Inspiron as an “oversized netbook” simply because it makes me appear more trendy.

According to an April survey by Changewave Research, one out of every four computers purchased in the next three months will be a netbook. These small, inexpensive laptop computers are quickly becoming the latest hip accessory. Unfortunately, their bare-bones computing abilities make them about as useful as a chihuahua in a handbag.

In a TECH.BLORGE interview, Lenovo analyst Matt Kohut admits that netbooks may not live up to the expectations of some consumers.

Initially people weren’t sure what to do with them. Retailers were saying, “here’s this new netbook PC,” and the average person picked one up and said, “oh wow, that’s small, maybe I can run Photoshop.” So, as an industry, we ended up with a lot of returns, because the functionality of what netbooks could do was not well communicated.

However, Kohut is upbeat about the future of netbooks. Second-generation netbooks are expected to have beefier computing power, run Windows 7 and offer 3G wireless capabilities, all while price points continue to drop.

So where is Apple’s sleek, user-friendly netbook alternative? Apple CFO Timothy Cook told investors not to hold their breath at the company’s quarterly-earnings call:

When I look at what is being sold in the netbook space today, I see cramped keyboards, terrible software, junky hardware, very small screens, and just not a consumer experience, and not something that we would put the Mac brand on, quite frankly.

However, Ars Technica isn’t quite buying Cook’s skepticism and points out that Apple always trashes an idea before doing it.

Even though they can’t play games or edit video and often strain when loading YouTube, netbooks continue to sell. Do these low-priced Internet browsers fulfill an actual need, or are customers simply blinded by impossibly low prices?

Read the rest of this entry...

Wolfram Alpha

Wolfram Alpha, does it ring a bell? Well maybe not but according to an article it has been making a buzz in the tech community. The said web site/service is said to launch on May 18 in the U.S.

Wolfram|Alpha looks like a search engine: it has a box where you type in a question or query terms. That’s about where the similarities end, though, because, unlike Google or Ask, Wolfram|Alpha is kind of like an enormous calculator. It takes your question and crunches out an entirely new answer, even if the answer isn’t something that’s been posted on the Web before.
Wolfram Alpha and Google are very different animals. Google is designed to help people find Web pages. It's a big lookup system basically, a librarian for the Web. Wolfram Alpha on the other hand is not at all oriented towards finding Web pages, it's for computing factual answers. It's much more like a giant calculator for computing all sorts of answers to questions that involve or require numbers. Alpha is for calculating, not for finding. So it doesn't compete with Google's core business at all. In fact, it is much more comptetive with the Wikipedia than with Google.On the other hand, while Alpha doesn't compete with Google, Google may compete with Alpha. Google is increasingly trying to answer factual questions directly -- for example unit conversions, questions about the time, the weather, the stock market, geography, etc. But in this area, Alpha has a powerful advantage: it's built on top of Wolfram's Mathematica engine, which represents decades of work and is perhaps the most powerful calculation engine ever built.

Wolfram Alpha also solves equations and shows the steps it took to do so, which will be of interest to high school students and math majors. Not into number crunching? If you live near the coast, you could type in “tides in ____” and find charts of tidal and lunar information. You could also graph that against other cities, which would be cool if you’re a surfer.

Wolfram Alpha is also interesting for academic queries. Type in “Internet users in Africa” and you’ll get the total number of Web users there — 51 million — as well as lists of the number of users by country plus graphs of this information. If you’re in the fisheries business, or if you’re an environmentalist, you could type in “fish produced in Italy versus France” to get an idea of how that sector is faring. The answer includes specifics, like how much of the fish crop was farmed versus what was captured. Such data could be used to argue policy points or to debate whether or not certain industries are sustainable.

Of course, questions abound. It remains to be seen just how smart Wolfram Alpha really is, or can be. How easily extensible is it? Will it get increasingly hard to add and maintain knowledge as more is added to it? Will it ever make mistakes? What forms of knowledge will it be able to handle in the future?

I think Wolfram would agree that it is probably never going to be able to give relationship or career advice, for example, because that is "fuzzy" -- there is often no single right answer to such questions. And I don't know how comprehensive it is, or how it will be able to keep up with all the new knowledge in the world (the knowledge in the system is exclusively added by Wolfram's team right now, which is a labor intensive process). But Wolfram is an ambitious guy. He seems confident that he has figured out how to add new knowledge to the system at a fairly rapid pace, and he seems to be planning to make the system extremely broad.

And there is the question of bias, which we addressed as well. Is there any risk of bias in the answers the system gives because all the knowledge is entered by Wolfram's team? Those who enter the knowledge and design the formal models in the system are in a position to both define the way the system thinks -- both the questions and the answers it can handle. Wolfram believes that by focusing on factual knowledge -- things like you might find in the Wikipedia or textbooks or reports -- the bias problem can be avoided. At least he is focusing the system on questions that do have only one answer -- not questions for which there might be many different opinions. Everyone generally agrees for example that the closing price of GOOG on a certain data is a particular dollar amount. It is not debatable. These are the kinds of questions the system addresses.

But even for some supposedly factual questions, there are potential biases in the answers one might come up with, depending on the data sources and paradigms used to compute them. Thus the choice of data sources has to be made carefully to try to reflect as non-biased a view as possible. Wolfram's strategy is to rely on widely accepted data sources like well-known scientific models, public data about factual things like the weather, geography and the stock market published by reputable organizatoins and government agencies, etc. But of course even this is a particular worldview and reflects certain implicit or explicit assumptions about what data sources are authoritative.

This is a system that reflects one perspective -- that of Wolfram and his team -- which probably is a close approximation of the mainstream consensus scientific worldview of our modern civilization. It is a tool -- a tool for answering questions about the world today, based on what we generally agree that we know about it. Still, this is potentially murky philosophical territory, at least for some kinds of questions. Consider global warming -- not all scientists even agree it is taking place, let alone what it signifies or where the trends are headed. Similarly in economics, based on certain assumptions and measurements we are either experiencing only mild inflation right now, or significant inflation. There is not necessarily one right answer -- there are valid alternative perspectives.

I agree with Wolfram, that bias in the data choices will not be a problem, at least for a while. But even scientists don't always agree on the answers to factual questions, or what models to use to describe the world -- and this disagreement is essential to progress in science in fact. If there is only one "right" answer to any question there could never be progress, or even different points of view. Fortunately, Wolfram is desigining his system to link to alternative questions and answers at least, and even to sources for more information about the answers (such as the Wikipeda for example). In this way he can provide unambiguous factual answers, yet also connect to more information and points of view about them at the same time. This is important.

It is ironic that a system like Wolfram Alpha, which is designed to answer questions factually, will probably bring up a broad range of questions that don't themselves have unambiguous factual answers -- questions about philosophy, perspective, and even public policy in the future (if it becomes very widely used). It is a system that has the potential to touch our lives as deeply as Google. Yet how widely it will be used is an open question too.

The system is beautiful, and the user interface is already quite simple and clean. In addition, answers include computationally generated diagrams and graphs -- not just text. It looks really cool. But it is also designed by and for people with IQ's somewhere in the altitude of Wolfram's -- some work will need to be done dumbing it down a few hundred IQ points so as to not overwhelm the average consumer with answers that are so comprehensive that they require a graduate degree to fully understand.

It also remains to be seen how much the average consumer thirsts for answers to factual questions. I do think all consumers at times have a need for this kind of intelligence once in a while, but perhaps not as often as they need something like Google. But I am sure that academics, researchers, students, government employees, journalists and a broad range of professionals in all fields definitely need a tool like this and will use it every day.

For now if you want to try using Wolfram Alpha you just have to wait until it's been launch since it's only available now for a "few individuals".

Read the rest of this entry...

Unix redirection with SUDO

Whenever you execute redirection of a command run with sudo, the redirection will be performed as the user calling sudo, not as the superuser (or user specified with the sudo -u flag).

Example:

$ sudo echo letter > /etc/papersize
bash: /etc/papersize: Permission denied

To perform redirection as the superuser (or sudo -u user), place the command in quotes after sh -c.

Example:

$ sudo sh -c 'echo letter > /etc/papersize'

Read the rest of this entry...

Howto: Print UNIX man pages

Some notes on printing manual pages:

To view "raw" man pages in nroff/troff format:
$ nroff -man manpage | less -s

Example:
$ nroff -man qtool.8 | less -s


To output a UNIX man page in a format suitable for printing, pipe the man page through col -b:
$ man command | col -b

To print a raw man page:
$ nroff -man qtool.8 | col -b | lpr

Read the rest of this entry...

Howto: Mounte a Linux LVM volume

Mounting a partition of type "Linux LVM" the same way you mount a partition using a standard Linux file system (e.g. ext2, ext3). You have to do certain steps to mount it as shown below:

# fdisk -l /dev/hda

Disk /dev/hda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 13 104391 83 Linux
/dev/hda2 14 19457 156183930 8e Linux LVM

# mount /dev/hda2 /tmp/mnt
mount: /dev/hda2 already mounted or /tmp/mnt busy

First, let's determine the volume group containing the physical volume /dev/hda2.

# pvs
PV VG Fmt Attr PSize PFree
/dev/hda2 VolGroup01 lvm2 a- 148.94G 32.00M
/dev/hdb2 VolGroup00 lvm2 a- 114.94G 96.00M

Next, let's list the logical volumes in VolGroup01.

# lvdisplay /dev/VolGroup01
--- Logical volume ---
LV Name /dev/VolGroup01/LogVol00
VG Name VolGroup01
LV UUID zOQogm-G8I7-a4WC-T7KI-AhWe-Ex3Y-JVzFcR
LV Write Access read/write
LV Status available
# open 0
LV Size 146.97 GB
Current LE 4703
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:2

--- Logical volume ---
LV Name /dev/VolGroup01/LogVol01
VG Name VolGroup01
LV UUID araUBI-4eer-uh5L-Dvnr-3bI6-4gYg-APgYy2
LV Write Access read/write
LV Status available
# open 0
LV Size 1.94 GB
Current LE 62
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:3

The logical volume I would like to "mount" (in purely the computing-related sense) is /dev/VolGroup01/LogVol00. The other logical volume is a swap partition.

# mount /dev/VolGroup01/LogVol00 /tmp/mnt

Read the rest of this entry...

Dual-homed iptables firewall

In this example, eth1 is the internal LAN interface and eth0 is the public Internet interface. All outbound and return traffic is allowed from both the internal LAN and the firewall itself. All incoming traffic originating from the Internet is dropped.

For remote administration via ssh, add a rule such as:

# Allow incoming ssh from work
/sbin/iptables -A INPUT -i eth0 -p tcp -s work_IP_address/32 --dport 22 -m state --state NEW -j ACCEPT

-----

#!/bin/sh

# Enable broadcast echo protection
echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts

# Disable source routed packets
for f in /proc/sys/net/ipv4/conf/*/accept_source_route; do
echo 0 > $f
done

# Enable TCP SYN cookie protection
echo 1 > /proc/sys/net/ipv4/tcp_syncookies

# Disable ICMP Redirect acceptance
for f in /proc/sys/net/ipv4/conf/*/accept_redirects; do
echo 0 > $f
done

# Don't send Redirect messages
for f in /proc/sys/net/ipv4/conf/*/send_redirects; do
echo 0 > $f
done

# Drop spoofed packets coming in on an interface, which if replied to,
# would result in the reply going out a different interface.
for f in /proc/sys/net/ipv4/conf/*/rp_filter; do
echo 1 > $f
done

# Log packets with impossible addresses
for f in /proc/sys/net/ipv4/conf/*/log_martians; do
echo 1 > $f
done

# Flush existing rules on INPUT, OUTPUT, FORWARD chains and nat table
/sbin/iptables --flush
/sbin/iptables -t nat --flush

# Unlimited traffic on the loopback interface
/sbin/iptables -A INPUT -i lo -j ACCEPT
/sbin/iptables -A OUTPUT -o lo -j ACCEPT

# Set the default policy to drop
/sbin/iptables --policy INPUT DROP
/sbin/iptables --policy OUTPUT DROP
/sbin/iptables --policy FORWARD DROP
/sbin/iptables -t nat --policy PREROUTING ACCEPT
/sbin/iptables -t nat --policy OUTPUT ACCEPT
/sbin/iptables -t nat --policy POSTROUTING ACCEPT

# Drop all invalid TCP state combinations
# First list of TCP state flags lists the bits to be tested
# Second list of TCP state flags lists the bits that must be set to match test

# All of the bits are cleared
/sbin/iptables -A INPUT -p tcp --tcp-flags ALL NONE -j DROP

# SYN and FIN are both set
/sbin/iptables -A INPUT -p tcp --tcp-flags SYN,FIN SYN,FIN -j DROP

# SYN and RST are both set
/sbin/iptables -A INPUT -p tcp --tcp-flags SYN,RST SYN,RST -j DROP

# FIN and RST are both set
/sbin/iptables -A INPUT -p tcp --tcp-flags FIN,RST FIN,RST -j DROP

# FIN is set without the expected accompanying ACK
/sbin/iptables -A INPUT -p tcp --tcp-flags ACK,FIN FIN -j DROP

# PSH is set without the expected accompanying ACK
/sbin/iptables -A INPUT -p tcp --tcp-flags ACK,PSH PSH -j DROP

# URG is set without the expected accompanying ACK
/sbin/iptables -A INPUT -p tcp --tcp-flags ACK,URG URG -j DROP

# Masquerade everything out eth0; used for dynamic IPs
/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

# Allow all outbound connections from LAN (eth1) to Internet (eth0)
# Allow only return traffic from those connections
/sbin/iptables -A FORWARD -i eth1 -o eth0 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
/sbin/iptables -A FORWARD -i eth0 -o eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow unlimited outbound and return traffic from firewall
/sbin/iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
/sbin/iptables -A OUTPUT -o eth0 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
/sbin/iptables -A OUTPUT -o eth1 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT

# Activate IP forwarding
/bin/echo 1 > /proc/sys/net/ipv4/ip_forward

# Save iptables rules
/sbin/service iptables save

Read the rest of this entry...

Top Web Picture Editors

Digital cameras are common nowadays and have also reduced their prices making it easier to have on. Gone were the days when you have to buy film and have them developed after use. One great thing about digital pictures is that you can easily preview it in your digital camera as well as edit the pictures you've taken.

Picture editing can be done using photo editors either free or for a price. Here I've collected some web tools for editing your picture online,for free or at most free.

1. Picnik - from their site: "Picnik makes your photos fabulous with easy to use yet powerful editing tools. Tweak to your heart's content, then get creative with oodles of effects, fonts, shapes, and frames." If you already use Flickr for your photos, you're only a click away from taking a Picnik.

2. Splashup- Splashup "is a powerful editing tool and photo manager. With all the features professionals use and novices want, it's easy to use, works in real-time, and allows you to edit many images at once." You'd wonder how something this extensive could be free - and it all runs within the browser.

3. FotoFlexer - FotoFlexer describes itself as "the world's most advanced online image editor." Using it, you can edit photos from Photobucket, MySpace, Facebook, Flickr, Picasa, and more.

4. Pixlr - "Pixlr is a free online photo editor; jump in and start: edit, adjust, filter. It's just what you imagine!" Alexa says it's the "fastest growing photo editor online." If these resources are starting to look similar to your eye, remember you don't have to use all of them - just pick the one that works best for your needs.

5. flauntR- flauntR is a suite that includes the ability to edit images, add effects, add text, and print the results on everything from mugs to posters. You might be overwhelmed with the options here, but... isn't that better than being underwhelmed?

6. Photoshop Express- Adobe's Photoshop was really the first image editor of its kind to herald in the age of digital photography, though it was designed for professionals and its endless options could be daunting (and bank- breakingly expensive) for the layman. Now, Photoshop.com is on the Web to help that very same layman -- for free or plus levels of membership -- edit, store, and share his photos. Lots of helpful tutorials promise to guide the casual user through editing pickles.

7. PiZap-- PiZap's tagline is "fun with photos made easy." It's another flavor of photo editor/storage center that, like flauntR, allows you to print the results to mugs, bags, and t-shirts. PiZap invites developers to check out its API for more personalization options (which is usually a good thing).

8. Aviary- Aviary lets you: "edit images, create mind-blowing effects, design logos, find colors, collaborate, and more. All you need is a Web browser." This particular brand looks to be the one to beat, as they have several different tools in development.

9. DrPic-- DrPic says it's "the easiest free online picture editor" and that it has "processed over five million pictures."

10. Cameroid -- Cameroid lets you "take crazy (or not) snaps with your webcam straight from your browser." I featured this one in a CNN.com live video a while ago, and it certainly deserves to be mentioned again. If you're looking for something else to do with your webcam, give it a cameroid.

There you go... the list is not absolute though I might have missed some others. Feel free to comment about those I missed.

Read the rest of this entry...

bulilit,bulilit, so cute

An TV ad that gives me LSS ( last song syndrome). Features a really cute kid Cha-cha Canete.

Read the rest of this entry...

Windows 7 RC free download

Windows 7 RC was officially released for public download this morning in the U.S. It's supposed to be a (mostly) finalized version of the new OS that successfully completed beta testing and could potentially be sold as the "final product".

The DVD image is at 2.36GB so you better have a lot of bandwidth if you plan on downloading it. It looks like Microsoft has solved its distribution issues after demand for the beta version of Windows 7 crippled a number of servers when it was released earlier in January.

The RC Windows 7 boasts of a quicker install, more polished and customizable and easier to use than the earlier builds.

I don't really like installing preproduction software but if you like doing so anyone is free to download and use Windows 7 RC until the temporary license expires June 2010.

Read the rest of this entry...

Edit PDF online

I was recently looking for a software to edit a PDF file I have without having to insstall anything on my computer ( it's a company computer shhh ^o^ ) and my search brought me to this online tool.

The online tool I'm talking of is PDFescape. It's a new way to open PDF files and escape from the typical software requirements for using the de facto document file format.

PDFescape is free online PDF editor:

Without required downloads or installations!
Without watermarks!
Without required registration! ( well registration is free anyway plus you can save whatever it is you are editing for future use)

With PDFescape, you can fill in PDF forms, add text and graphics, add links, and even add new form fields to a PDF file.

This online tool is in beta but works really well, I hope it stays free ^_^.

Read the rest of this entry...

iostat disk throughput script

Here's a script that can be used to visualize iostat disk throughput which will normally be in columns...

#!/usr/bin/perl -w

use strict;
my $bar_char="=";
my @data;
my $current_read;
my $current_write;
my @dev_len;

my $iostat_lines=0;
open (IOSTATTEST, "iostat -x|");
while(){$iostat_lines++};
print "iostat lines: $iostat_lines\n";
system("clear");

my $current_line=0;
open (IOSTAT, "iostat -x 5|");
while (){
if ($current_line == $iostat_lines){system("clear"); $current_line=0;}
if ($_ =~/\d/){
@data=split(/\s+/,$_);
$current_read=bar($data[3]);
$current_write=bar($data[4]);
@dev_len=split(//,$data[0]);
my $space=" ";
if ($#dev_len == 2){$space=" "};
print "$data[0]$space kb/s r $current_read $data[3]\n";
print "$data[0]$space kb/s w $current_write $data[4]\n";
}
$current_line++;
}

sub bar(){
my $in=$_[0];
my $char="=";
my $blength=($in / 100);
my $rval=$char;
while ($blength > -1){
$rval="$rval"."$char";
$blength=$blength - 1 ;
}
return $rval;
}

Sample output will be as such:

dad0 kb/s r =========== 848.4
dad0 kb/s w ==================== 1764.1
dad1 kb/s r ============ 935.8
dad1 kb/s w ==================== 1789.7
md0 kb/s r == 0.0
md0 kb/s w == 0.0
md1 kb/s r == 0.0
md1 kb/s w == 0.0
md2 kb/s r == 0.0
md2 kb/s w == 0.0
md10 kb/s r == 0.0
md10 kb/s w == 0.0
md11 kb/s r == 0.0
md11 kb/s w == 0.0
md12 kb/s r == 0.0
md12 kb/s w == 0.0
md30 kb/s r ==================== 1784.2
md30 kb/s w ==================== 1763.2
md31 kb/s r =========== 848.4
md31 kb/s w ==================== 1763.2
md32 kb/s r ============ 935.8
md32 kb/s w ==================== 1788.8
md50 kb/s r == 0.0
md50 kb/s w === 0.8
md51 kb/s r == 0.0
md51 kb/s w === 0.8
md52 kb/s r == 0.0
md52 kb/s w === 0.8

credits to www.networkz.ch for the scrips.

Read the rest of this entry...

SED examples

SEDis a Unix utility which parses text files and implements a programming language which can apply textual transformations to such files. It reads input files line by line (sequentially), applying the operation which has been specified via the command line (or a sed script), and then outputs the line. Below are some examples on how SED is normally used.

replace a string, save in a new file:

sed -e 's/VMS/Unix/g' file > file.new

delete all lines starting with "Novell":

sed -e '/^Novell/d' file > file.new

Read the rest of this entry...

View /var/adm/messages in order

Usually the messages file in Solaris would be rotated and appended with a number in the end. I usually use this command to view the files in order.

The command I use is this:

#ls -1A /var/adm/mess* | sort -r | xargs cat | less

With this command all message files can be viewed at once, in the correct chronological order.

Read the rest of this entry...

Firefox 3 performance sucks

I have been using firefox as an alternative browser from the start happily up until they released firefox 3.x.x. I don't know about you guys who also use firefox as a browser but here is why I think the new firefox 3 sucks and READ : IT SUCKS BIG TIME at least inmy opinion.

The first time I upgraded to firefox 3 I immediately noticed how long it takes to start-up and after a lot of time using it I got irritated since the browser would usually crash on me. Finally after some days or weeks they released an update to it but still with the same outcome; it HANGS. I can deal with the slow start-up but not with hangs/crashes, I hate seeing the "not responding" at the firefox 3 bar.

And for those thinking it might be my computer. well I'm already using a high-end pc and laptop with 4G of memory so hardware is certainly not an issue.

I'm still looking forward that the firefox team can fix the crashes but for now I think O'll be using Chrome or IE7 instead.

If only Chrome has a seobook toolbar available for it.

Well, just my 2 cents

Read the rest of this entry...