Showing posts with label work. Show all posts
Showing posts with label work. Show all posts

Thursday, October 25, 2018

Kindie Review: New Sun Moon, Bangsar South

Thursday, October 25, 2018 1 Comments

Here is my third review for a kindie, also in Bangsar South. I called the number posted on their Facebook which looks like direct translation from Chinese to English. Never mind the crappy name, their reviews were really good. The lady sounded busy, so she told me to pop in tomorrow morning.

Here's what I managed to glean from her over the phone. She was clear that the fees et al were for 2018 only.

Full day programme

8am till 7pm

RM 1370 per month

Food provided for breakfast, lunch, and post nap

Twice naptime; once after lunch and the second after shower (when is this I don't know)

Half day programme

8am till 12.30pm

RM 1170 per month

Deposit

to be updated tomorrow

What is provided

to be updated tomorrow

What you need to pay for 

to be updated tomorrow

Kindie Review: WHITELODGE PRESCHOOL, Bangsar South

Thursday, October 25, 2018 3 Comments

Pretty much mixed reviews about this school. A lot of expats live in my area and the Indian expats stay out of this kindie because of their laissez faire approach. There were some pretty bad reviews about this place, and the nastiest one was about how the teachers are engrossed in their phones and let children cry or fight among themselves. Left unsupervised. *the horror* I can't find this damning review online anymore, it must have been removed or not showing up on my search with VPN.

However, lately I have been walking back home everyday from work and I can see a handful of parents walking with their child in tow; coming out of this kindie. Sometimes I can see their staffs walking to the LRT.

So if these parents are putting their kids in there-surely it must be doing something right-to a degree? Or has the management changed? For parents that are unaware; after 3-4 years or running a kindie, the owner is allowed to "sell it off" to someone for a higher fee. By selling I mean the license, the contract, the right to use the kindie name; the whole works. It is exactly like selling an AP permit. The latter being illegal but the former; legal. There is plenty money to be made for such sales. But I believe teaching is a calling-there are some good kindies out there. I just have to find it.

Anyways let us backtrack to White Lodge. I called them last Friday to inquire and someone followed up with me on Monday morning with a phone call and answered my flurry of emails.

Here are some basic info gleaned from White Lodge. Please note that as I am looking for a full day programme for my son, I only inquired about their full day programme.

Time

8.00 am till 6.30 pm

Terms

4 terms a year
July –September, October – December, Jan – March, April – June

Fees

2 day a week per term - RM 4620

3 day a week per term – RM 4920

5 day a week per term – RM 5220

Monthly breakdown for 5 days full day programme: RM1740

Registration fee: RM 350

Refundable deposit: RM 500

What is provided

breakfast, lunch, post-nap snack

nap at 1pm till 3pm

no uniform

What you have to pay for

field trips once every term with parents

activity; if organised in a month

They have an upcoming UN Day Celebration and orientation day on the 27th October 2018 and they have invited me to join them if i wish to. You could email them to ask about it.

I will reserve my judgement until I go there myself.

Kindie Review: The Children's House

Thursday, October 25, 2018 0 Comments

They are made famous by their red and white picket fence bungalow strategically placed in Bangsar and around Klang valley.

Of course, on their admission page you will not find any reference to fees.

All you need is a phone call to the number provided on their website (my emails were unanswered) which will be directed to their HQ customer service (wherever that is)

Today I took the leap and decided to call the number listed. After 3 attempts, someone finally picked up and I got the lay down on their fees and structure.

I inquired about 4 year old since my son is 4.

They have 3 programmes. The quarter day, half day and full day.

  • quarter day ends at 12pm
  • half day ends at 3pm (I am interested in this)
  • full day ends at 6pm (not all their branches provide full day; only a selected few)
Terms

4 terms a year

Time schedule:

9am till 3pm

Fees (for 4 year old, half day programme)

RM 7960 deposit (refundable upon 4 months notice of your child leaving the kindie)

RM 7960 per term fee

RM 1180 one-off stationary fee

RM 500 registration fee promo

What is provided

Breakfast and lunch; no nap time

1 pair of uniform (only mandatory for field trips)

Snack such as juice and biscuits if your child happens to be hungry

What you still have to pay for

field trips costs

outdoor activity planned by the teachers

As you can see, the whole fees structure put me off. It's not meant for middle class parents. Rather; the school aims everything stated on their website's goals for children meant to be coming from rich backgrounds. I am a poor fuck, this won't do for me. Boo Hoo

*Registration closes at 31st October 2018

Thursday, July 26, 2018

Linux: Configuring NTP on Ubuntu 18.4 host and RHEL 7.1 VM guest running on Oracle VirtualBox

Thursday, July 26, 2018 0 Comments
I almost never expected to get this going at all. I had been tinkering the whole day and eventually with a little help from my colleague I finally managed to get it running.

I have on my Lenovo laptop; installed Ubuntu 18.4. It boots on Ubuntu, nothing else. I had completely broken down everything on my BIOS to get Ubuntu running. Couple of days back I had installed Oracle VM Virtualbox. I then added a RHEL 7.1 VM/guest on the Virtualbox and named it apollo. For my VirtualBox network settings, I used Bridged Adapter. I also disabled the Promiscuous Mode. Here's a picture of my VM's network settings.



I then proceeded to configure apollo's in accordance to the company's requirement. Halfway done, I came across the NTP config setting. The document I was referring to is outdated at best. So in my own way this is what I did to get my laptop (host) act as an NTP server and my Virtualbox VM (guest) act as NTP client.

On my laptop running Ubuntu 18.4 (host/homer)

Until recently, most network time synchronization was handled by the Network Time Protocol daemon or ntpd. This service connects to a pool of other NTP servers that provide it with constant and accurate time updates.

Ubuntu's default install now uses timesyncd instead of ntpd. timesyncd connects to the same time servers and works in roughly the same way, but is more lightweight and more integrated with systemd and the low level workings of Ubuntu.

We can query the status of timesyncd by running timedatectl

root@homer:~# timedatectl

Local time: Rab 2018-07-25 11:45:45 +08

Universal time: Rab 2018-07-25 03:45:45 UTC

RTC time: Rab 2018-07-25 03:45:45

Time zone: Asia/Kuala_Lumpur (+08, +0800)

System clock synchronized: yes

systemd-timesyncd.service active: yes------------------->look here


If timesyncd isn't active, turn it on with timedatectl:

# timedatectl set-ntp on
Switching to ntpd

Though timesyncd is fine for most purposes, some applications that are very sensitive to even the slightest perturbations in time may be better served by ntpd, as it uses more sophisticated techniques to constantly and gradually keep the system time on track.

Before installing ntpd, we should turn off timesyncd:

# timedatectl set-ntp no

root@homer:~# timedatectl

Local time: Rab 2018-07-25 11:30:59 +08

Universal time: Rab 2018-07-25 03:30:59 UTC

RTC time: Rab 2018-07-25 03:30:59

Time zone: Asia/Kuala_Lumpur (+08, +0800)

System clock synchronized: yes

systemd-timesyncd.service active: no------------------->look here


We can now install the ntp package with apt:

# apt-get install ntp

# systemctl enable ntp


root@homer:~# service ntp status
● ntp.service - Network Time Service
Loaded: loaded (/lib/systemd/system/ntp.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Wed 2018-07-25 18:00:05 +08; 57s ago
Docs: man:ntpd(8)

Main PID: 9177 (code=exited, status=0/SUCCESS)

Jul 25 18:00:05 homer ntpd[9177]: ntpd exiting on signal 15 (Terminated)
Jul 25 18:00:05 homer ntpd[9177]: 10.101.101.103 local addr 10.101.101.103 ->
Jul 25 18:00:05 homer systemd[1]: Stopping Network Time Service...
Jul 25 18:00:05 homer ntpd[9177]: 36.3.117.150 local addr 10.101.101.103 ->
Jul 25 18:00:05 homer ntpd[9177]: 129.250.35.250 local addr 10.101.101.103 ->
Jul 25 18:00:05 homer ntpd[9177]: 45.125.1.20 local addr 10.101.101.103 ->
Jul 25 18:00:05 homer ntpd[9177]: 167.99.64.239 local addr 10.101.101.103 ->
Jul 25 18:00:05 homer ntpd[9177]: 91.189.89.198 local addr 10.101.101.103 ->
Jul 25 18:00:05 homer ntpd[9177]: 91.189.91.157 local addr 10.101.101.103 ->
Jul 25 18:00:05 homer systemd[1]: Stopped Network Time Service.

root@homer:~# service ntp start

root@homer:~# service ntp status

● ntp.service - Network Time Service

Loaded: loaded (/lib/systemd/system/ntp.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2018-07-25 18:01:18 +08; 2s ago
Docs: man:ntpd(8)

Process: 10071 ExecStart=/usr/lib/ntp/ntp-systemd-wrapper (code=exited, status=0/SUCCESS)
Main PID: 10079 (ntpd)
Tasks: 2 (limit: 4915)
CGroup: /system.slice/ntp.service

└─10079 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 126:131

Jul 25 18:01:18 homer ntpd[10079]: Listen normally on 2 lo 127.0.0.1:123
Jul 25 18:01:18 homer ntpd[10079]: Listen normally on 3 wlp2s0 10.101.101.103:123
Jul 25 18:01:18 homer ntpd[10079]: Listen normally on 4 tun0 192.168.193.49:123
Jul 25 18:01:18 homer ntpd[10079]: Listen normally on 5 lo [::1]:123
Jul 25 18:01:18 homer ntpd[10079]: Listen normally on 6 wlp2s0 [fe80::df55:6242:fc26:a259%3]:123
Jul 25 18:01:18 homer ntpd[10079]: Listen normally on 7 tun0 [fe80::cee:c8ae:65cd:5d19%4]:123
Jul 25 18:01:18 homer ntpd[10079]: Listening on routing socket on fd #24 for interface updates
Jul 25 18:01:18 homer ntpd[10079]: Listen for broadcasts to 10.101.101.255 on interface #3 wlp2s0
Jul 25 18:01:19 homer ntpd[10079]: Soliciting pool server 103.245.79.18
Jul 25 18:01:20 homer ntpd[10079]: Soliciting pool server 202.65.114.202

You can query ntpd for status information to verify that everything is working:

root@homer:~# ntpq -p

remote refid st t when poll reach delay offset jitter

==============================================================================

0.ubuntu.pool.n .POOL. 16 p - 64 0 0.000 0.000 0.000
1.ubuntu.pool.n .POOL. 16 p - 64 0 0.000 0.000 0.000
2.ubuntu.pool.n .POOL. 16 p - 64 0 0.000 0.000 0.000
3.ubuntu.pool.n .POOL. 16 p - 64 0 0.000 0.000 0.000
ntp.ubuntu.com .POOL. 16 p - 64 0 0.000 0.000 0.000
+203.95.213.129 195.66.241.10 2 u 18 64 3 73.523 -59.709 132.626
+timpany.srv.jre 133.11.204.98 2 u 17 64 3 200.323 -104.51 182.499
+ntp.hkg10.hk.le 130.133.1.10 2 u 16 64 3 100.148 -85.168 123.225
-dadns.cdnetwork 216.239.35.12 2 u 13 64 3 97.346 -67.509 138.111
#203.217.204.135 125.142.117.152 2 u 17 64 3 532.743 -181.34 247.584
+82.200.209.236 89.109.251.22 2 u 16 64 3 280.073 -51.717 120.860
+ntp.nic.kz .SHM. 1 u 16 64 3 303.338 -46.429 123.972
*45.125.1.20 (45 223.255.185.2 2 u 16 64 3 59.258 -67.442 135.406
+ntp-sin-02.no-s 103.1.106.69 2 u 17 64 3 58.434 -52.011 136.924
+ntp-a2.nict.go. .NICT. 1 u 20 64 3 200.409 -105.70 181.385
#185.105.186.198 193.93.167.239 2 u 16 64 3 308.142 -92.379 137.911
+jiro.paina.net 131.113.192.40 2 u 17 64 3 113.411 -55.281 137.586
#ntp.gnc.am 195.43.74.123 2 u 14 64 3 309.324 -85.812 128.094
#120.25.115.19 10.137.53.7 2 u 18 64 3 387.921 -51.942 141.329
#ntp.uii.net.id 133.243.238.243 2 u 15 64 3 54.574 -63.613 139.148
pugot.canonical 17.253.34.125 2 u 22 64 3 533.729 -164.53 246.341
#masao.paina.net 131.113.192.40 2 u 16 64 3 366.324 -92.037 124.93


Activate broadcast on my Ubuntu host. For this case, edit your /etc/ntp.conf file and configure the lines below:

#broadcast
#broadcastclient------------------->comment this


The broadcasting IP address is your host's IP address, NOT YOUR VM IP address.

Make sure that your NTP port is opened:

# iptables -A INPUT -p udp --dport 123 -j ACCEPT

# iptables -A OUTPUT -p udp --sport 123 -j ACCEPT


Check if your port is open with the nmap scanner:

root@homer:~# nmap -p123 -sU -P0 localhost

Starting Nmap 7.60 ( https://nmap.org ) at 2018-07-26 10:37 +08
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00011s latency).

PORT STATE SERVICE
123/udp open ntp

Nmap done: 1 IP address (1 host up) scanned in 0.30 seconds

Restart ntp service

# service ntp restart

Check if NTP is syncing properly with ubuntu's NTP servers:

root@homer:~# ntpstat
synchronised to NTP server (129.250.35.251) at stratum 3

time correct to within 192 ms
polling server every 1024 s


There are a lot of limitations on Ubuntu 18 on a service checking. For one, chkconfig is no longer available for download. I was not able to download and install sysv-rc-conf and I was instead given the option to install init-system-helpers but I wasn't going to learn something new so I just resorted to service, update-rc.d command and systemctl commands

To check if ntp has been enabled at boot, run the service command, and look for ntp:

root@homer:~# service --status-all

[ + ] acpid
[ - ] alsa-utils
[ - ] anacron
[ + ] apparmor
[ + ] apport
[ + ] avahi-daemon
[ + ] bluetooth
[ - ] console-setup.sh
[ + ] cron
[ + ] cups
[ + ] cups-browsed
[ + ] dbus
[ - ] dns-clean
[ + ] gdm3
[ + ] grub-common
[ - ] hwclock.sh
[ + ] irqbalance
[ + ] kerneloops
[ - ] keyboard-setup.sh
[ + ] kmod
[ - ] lvm2
[ + ] lvm2-lvmetad
[ + ] lvm2-lvmpolld
[ + ] network-manager
[ + ] networking
[ + ] ntp----------------------------->should be a + sign
...

If it is a negative (-) sign then enable ntp to run at boot up with the other rc scripts

root@homer:~# update-rc.d ntp defaults

Enable ntp on all run levels

root@homer:~# update-rc.d ntp enable 2 3 4 5

And you're pretty much done. Let's move on to our guest/client/VM running on our Ubuntu host.

On my Virtual Machine running RHEL 7.1 (guest/apollo)

First things first, if not yet installed, please use yum to install ntp and ntpdate

# yum install ntp ntpdate

Once installed and before we start the service, we want to make sure that the ntpd service starts up automatically on boot and on all run levels so that we can maintain accurate time.

[root@apollo ~]# systemctl list-unit-files | grep ntpd

ntpd.service enabled
ntpdate.service enabled

[root@apollo ~]# chkconfig --level 345 ntpd on
Note: Forwarding request to 'systemctl enable ntpd.service'.
ln -s '/usr/lib/systemd/system/ntpd.service' '/etc/systemd/system/multi-user.target.wants/ntpd.service'

Edit the NTP config file at /etc/ntp.conf:

[root@apollo ~]# more /etc/ntp.conf
driftfile /var/lib/ntp/drift
restrict default nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict ::1
restrict 10.101.101.0 mask 255.255.255.0 nomodify notrap
server 10.101.101.103-------------------------->this is my Ubuntu host's IP address
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
disable monitor

Allow port 123 for NTP:

[root@apollo ~]# iptables -A OUTPUT -p udp --dport 123 -j ACCEPT

[root@apollo ~]# iptables -A INPUT -p udp --sport 123 -j ACCEPT

[root@apollo ~]# service ntpd start
Redirecting to /bin/systemctl start ntpd.service
Sync NTP with homer (the IP address below is my Ubuntu host's)

[root@apollo ~]# ntpdate -u 10.101.101.103
25 Jul 18:14:03 ntpdate[2478]: adjust time server 10.101.101.103 offset -0.025663 sec

[root@apollo log]# ntpq -c lpeer
remote refid st t when poll reach delay offset jitter

==============================================================================
*10.101.101.103 133.243.238.243 2 u 19 64 37 0.117 -4.281 15.797


I found that apollo's NTP still wasn't syncing with homer, so I ran ntpdate again with localhost:

[root@apollo ~]# ntpstat
unsynchronised

time server re-starting
polling server every 8 s

[root@apollo ~]# ntpdate -u 127.0.0.1
25 Jul 18:19:34 ntpdate[2481]: adjust time server 127.0.0.1 offset 0.000000 sec

[root@apollo ~]# ntpstat
synchronised to NTP server (10.101.101.103) at stratum 4

time correct to within 111 ms
polling server every 64 s


Once you have done that, update your VM's hardware clock with the system clock:

[root@apollo ~]# hwclock --systohc

[root@apollo ~]#

The end! My VM's NTP time is now in sync with my laptop's!

Friday, January 5, 2018

Linux: Script to test IPs

Friday, January 05, 2018 1 Comments
So today I was sitting free, and the sysadmin gave me a list of IPs to ping. Being the lazy person I am, I decided to look online for ready-made scripts to ping IPs and list out the results pingable (UP) unpingable (DOWN)

The first search result had a lot of samples to work with and I picked the best:

Of course the sysadmin helped me tweak the script:



Bear in mind that an IP is not pingable, it may take several seconds before the script moves on to the next IP, so the script in general may time some time to run and output its results, so if you are testing a 100 IPs, just pipe the results to a file on the system and run the job in the background so you can check the results when you are free later.

I am not cruel, here is the script for you to copy paste it and run on your Linux (tested on RHEL)

#! /bin/bash

for ip in XXX.XXX.XXX.{XXX..XXX}; do  # for loop and the {} operator
    ping -c 1 $ip > /dev/null 2> /dev/null  # ping and discard output
    if [ $? -eq 0 ]; then  # check the exit code
        echo "${ip} is up" # display the output
        # you could send this to a log file by using the >>pinglog.txt redirect
    else
        echo "${ip} is down"
    fi
done

Thursday, October 26, 2017

Friday, August 11, 2017

UNIX : Changing swap value on HP-UX and AIX

Friday, August 11, 2017 1 Comments
I don't like messing with a prod server's swap value unless there is a direct request from a customer

Here's how you do it on HP-UX and AIX. I trust you have your CR raised

HP-UX

The default value for swappiness is 60. You can alter it temporarily (until you next reboot) by typing as root

echo 50 > /proc/sys/vm/swappiness

If you want to alter the swap value change permanently then you need to change the vm.swappiness parameter in the /etc/sysctl.conf file

Step 1: check what is the current value for swap

sudo cat /proc/sys/vm/swappiness

(It should give a number 60 here)

Step 2: change to a lower number

sudo sysctl -w vm.swappiness=5

Step 3: check again

sudo cat /proc/sys/vm/swappiness

(Now, it should give a number 5 here)

Step 4: append the line vm.swappiness=5 in file /etc/sysctl.conf using vi editor

AIX

Things are easier on AIX with the use of smitty. I used AIX version 5.3

1. Check the current paging and swapspace value

# lsps -a

# pstat -s

2. Run smitty and make changes on the fly

# smit pgsp-----to increase/decrease paging

3. Check paging and swapspace value again

# lsps -a

# pstat -s

4. To add paging space

# smit lvm---paging space---add

or

#smit mkps

UNIX: Solving Stale PP Error after VIO2 shutdown in AIX

Friday, August 11, 2017 0 Comments
I had a customer that had 2 VIOs setup for their LPARs. When VIO2 was shutdown/powered back on, as expected the rootvg on the server showed "stale partition" or "stale PP" error

To remediate this problem the rootvg on client server needs to be varyon, but then we also have a second problem where the sysdump is holding onto the rootvg; thus disabling us to perform varyon

So in order to do a VG varyon, we need to release rootvg from the clutches of sysdump

I trust you have raised a CR for this task

1. The first step is to make note of the sysdump location

@stepper:/# sysdumpdev
primary              /dev/lg_dumplv
secondary            /dev/lg_dumplvII
copy directory       /var/adm/ras
forced copy flag     TRUE
always allow dump    TRUE
dump compression     ON

2.  Change the sysdumpdev to a temporary "null" device

@stepper:/# sysdumpdev -p /dev/sysdumpnull
primary              /dev/sysdumpnull
secondary            /dev/lg_dumplvII
copy directory       /var/adm/ras
forced copy flag     TRUE
always allow dump    TRUE

dump compression     ON

@stepper:/#  sysdumpdev -s /dev/sysdumpnull
primary              /dev/sysdumpnull
secondary            /dev/sysdumpnull
copy directory       /var/adm/ras
forced copy flag     TRUE
always allow dump    TRUE

dump compression     ON


3. Perform the varyon

# varyonvg rootvg

4.  Sync rootvg

# syncvg -v rootvg

(NOTE: will probably give error "Unable to synchronize logical volume")

5. Check the vg and make sure all PV are active

# lsvg -p rootvg

6. Set the sysdumpdev back to their original values (from step 1)

# sysdumpdev -p /dev/hd7

# sysdumpdev -s /dev/hd71

7. Check that everything is ok now

# lsvg rootvg

Tuesday, August 8, 2017

UNIX: Problems and Solutions to Running Nickel on HP-UX

Tuesday, August 08, 2017 0 Comments
Back then whenever a server crashed, one of the first things I used to do was to run nickel script to collect all kind of system logs so us engineers can start investigation. Nickel stands for Network related Information Collector, Keeper and Elaborator.

It is free, sometimes located in the HP-UX machine itself, or maybe in your sharepoint. Download and save it into the server and run the sh script

But what I am going to show here is the problems that usually arise with nickel (imagine running a stress test for a heart patient) and the solution

Tested on older versions of HP-UX

I trust you have the necessary tools and permissions to run nickel

Transferring nickel file from Windows to UNIX via ftp

1. Unzip latest nickel.zip in PC (Windows)

2. If /opt/contrib/bin does not exist on client server then create a directory path 

     # mkdir -p /opt/contrib/bin

3. FTP nickel from Windows PC to HP-UX server in "BINARY" mode, into /opt/contrib/bin

4. In the HP-UX server, go into the said directory

     # cd /opt/contrib/bin 

5. Change permission of the file

     # chmod 555 nickel 

6. Run nickel script

     # ./nickel 

7. A file: nickel.yourhostname.tar.gz will be created.

8. FTP in "BINARY" mode, nickel.yourhostname.tar.gz to Windows PC

The reason I am stressing about FTPing in Binary mode is because by default most FTP software uses ASCII mode and this is generally corrupt the tarball file, hence the Binary mode to transfer

====================================================================
What to do if nickel script appears to hung while running (i.e running for 30 mins is one of it)
====================================================================

Well, if nickel is not giving any output, then it may have hung while running. I have had this many times, not a reason to worry or panic. Here's what I used to do

Solution 1

1. Exit nickel session with Ctrl + C

2. Issue following commands

     # /sbin/init.d/diagnostic stop
     # /sbin/init.d/diagnostic start

3. Run Nickel again

     # cd /opt/contrib/bin
     # ./nickel

Solution 2

This is a solution to yet another problem that may arise after attempting Solution 1 where nickel again hangs ar *stm*

1. Telnet another session to the said server

2. Run the following commands

     # ps -ef | grep nickel
     # kill -9
     # kill -9

   Repeat above step if necessary

Solution 3

Because nickel is not perfect or God sent...

Possible causes for error "There is not enough memory available now" while running nickel

1. Server was too busy at that time

2. While the script does # cat  of big *.log files or when it is collecting network log netfmt

3. You may may have noticed some logs collected in nickel appear blank (in this case the process is not completed, hence the blank logs)

4. OLD STM , the query for STM info is slow, believe me

Solution to memory shortage error:

1. Backup & nullify the logs (if it is too big) and re-run nickel

     # >

2. Check the memory utilization of servers by running the following commands

     # top

     # glance -m

3. Run the nickel when the server is free

*I was using a very old version of nickel from 2007/2008

Saturday, July 29, 2017

UNIX : Adding 2 LUNs to VIO servers for AIX client server

Saturday, July 29, 2017 0 Comments
This is a tutorial (adapted) to add 2 LUNs to 2 VIO servers for use by AIX client server gummybears

I trust you have raised a Change Request, and a request to SAN team to assign LUNs to both VIO servers.
The number of LUNs depend on the client server gummybears' LVM configuration.
In most cases 4 LUNs is assigned and these will be striped on gummybears for FS extension

1. Adding 2 LUNs to vio1 and vio2

(These steps are for vio1 and should be repeated on vio2)

Login as padmin on vio1

Switch to oem mode

# oem_setup_env

Pick up new path and environment

# su -

Scan for disks

# cfgmgr

2. Identify the new disks. For security reasons I have removed the original server name and replaced it with "gummybears" Of course I did a poor editing job but you get the picture


3. There is a crafty little script someone has created and stored in /usr/local/scripts to set the reserve policy of the disks (I will share the content of the scripts at the end)

So I just run the script...

4. Make the pvid of the disks visible on both vios


5. We then identify the vhost for client gummybears. It will have two vhosts on each vio server, one for rootvg disks and another data VGs. Identify the data VG vhost


6. From the output above, it appears that vhost4 is used for rootvg. We don't touch that


From the output above, it appears that SAN LUNs are assigned to vhost12. We use this vhost to make virtual target devices

There is another crafty script that is used to generate mkvdev commands (I will share the contents later)  Once done, repeat steps 1 - 6 on vio2

#########################################################################

We now go about making the LUNs available on client server gummybears

1. Make note of current disks

# lspv

2. Scan for disks allocated from VIO

# cfgmgr

3. Identify new disks

# lspv



4. We then run the crafty script in client server gummybears to set disk health check interval (the output below had been truncated for clarity)



5. We then add the disks to the VG

# extendvg vgapp hdisk21 hdisk22

6. Set the disks' path priorities (crafty script does the job, output truncated for clarity)


Done!

###############################################################################

And here are the nifty scripts (they're not mine, found online somewhere)

1. Script to set reserve policy on disks on vio1


2. Script to generate mkvdev commands on vio1

3. Script to run on client gummybears to set the disk health check interval

4. Script to set disk path priority on client gummybears

UNIX: AIX LVM Best Practice

Saturday, July 29, 2017 0 Comments
Here are the general rules of thumb when creating a new FS on AIX

1. Understand the requirement
  • What is the FS size requested?
  • Do we have free space in the VG?
  • Does the LV need to be striped and the number of disks to be striped are?
  • What is the LV policy of the VG?

2. What kind of FS need to have CIO?

Normally in most of the user defined FS, except AIX default system FS

3. What FS needs to be configured as striping?

A tough question, but most of the user defined FS require to striped except AIX's default system FS

Where striping is needed, the recommendation of LV configuration is usually "poor man" striping instead of "strict" striping (a lot of old AIX servers still use "strict" striping but the trend has move to "poor man"striping)

4. Why this "poor man" striping policy over "strict" striping policy?

Simply because it provides more flexibility in managing the FS when it comes to LV extension as compared with the latter

And the second most important reason is it makes a Unix system admin's life a hell lot easier to live!

5. A "poor man" striping has the following characteristics

If you look at the PP distribution you will know if it is striped or not


In comparison, a strict striping has the following key characteristics


6. An example on how to configure poor man striping LV with CIO on a production server

6.1 First we detect the LUN

# cfgmgr and # lspv -L commands

6.2 We then change the PV attribute

# chdev -l hdiskX -a 'hcheck_interval=0 rw_timeout=60'

6.3 Then we add the disks to the VG

# extendvg vkandi hdiskX hdiskX hdiskX hdiskX

6.4 Create poor man striping (stripe to 4 disks)

# mklv-e x -t jfs2 -b n -y lvkandi vgkandi 1000 hdiskX hdiskX hdiskX hdiskX

This will create an LV lvkandi in VG vgkandi with 1000 Logical Partitions

6.5 We then create a JFS on the LV defined

# crfs -v jfs2 -d lvkandi -m /hedkandi -A yes -p rw -a agblksize=4096

6.6 Enable CIO option

#chfs -a options=cio,rw /hedkandi

6.7 Mount the FS

# mkdir /hedkandi

# mount /hedkandi

# df -h /hedkandi

6.8 Verify that the LV is mounted as CIO

# mount | grep lvkandi

7. To check if a FS is configured as CIO


8. To check if a FS is mounted as CIO

9. To enable CIO on a FS


10. To change an existing LV policy from strict to poor man striping or vice versa

There isn't a direct solution that I know of, and my suggestion is that you do a backup and recreate the LV with the desired striping

###############################################################################

Now that we are done with LV, we go to VG creation guideline

It is recommended to configure a Vg as a Scalable VG with an exception for rootvg

A scalable VG will support up to 1024 Physical Volume

1. Creating a scalable VG

First we identiy the PV to use for the VG creation

Create scalable VG with the following commands

# smitty mkvg---------->Add a Scalable Volume Group

Fill in the "Volume Group Name" "Physical Volume Names" and leave the rest as default

2. How to determine a VG type


Adapted from notes from my former workplace

Friday, July 28, 2017

UNIX : Forcibly Unmounting NFS filesystems on HPUX

Friday, July 28, 2017 0 Comments
There comes a time when you have to do what you have to do

When you have tried the usual umount command to unmount a NFS but it has failed due to many reasons (user using the NFS, processes run by user using the NFS, waiting didn't do it, etc)

Here are some alternative steps to unmounting an NFS



1. Attempt to unmount with the usual umount command failed

2. Use the fuser command to find out process IDs, user running/using the NFS

3. Do a ps to find the offending processes

4. Do a fuser with kill option to kill the processed holding up the NFS

5. Unmount the NFS again, now that all other processes have been terminated

And here's an important message when running fuser command


LINUX : Shrink JFS2 File System on LINUX

Friday, July 28, 2017 0 Comments
I once had a request when an engineer incorrectly extended a different file system than requested. These kind of human error always happen because hey, we're not all that perfect (and sometimes misreading and misunderstanding the user's request)

So I had to correct his mistake and get back to the customer because I was on great terms with the team. Yups, that happens too

Anyhow, here goes

What happens if we resized the wrong FS??

You start with the problem

Many thanks, but there is one problem, as per my initial request I needed /oracle/dat4 not /oracle/dat3

Make sure no one is using/accessing the file systems and resize2fs is installed on the server and the FS was created with JFS2

# umount /oracle/dat3

# resize2fs /dev/vg04/lvol2 20464M (this is the original size before we went about clobbering it)

# lvreduce -L 20464M /dev/vg04/lvol2

# mount /oracle/dat3

I had forgotten exactly which OS this Linux machine was running on. Hence the title Linux

UNIX : Replace/Mirror disks with no downtime on HPUX

Friday, July 28, 2017 0 Comments
This tutorial is simple, to replace disks on a server, without a downtime (live) , so mirroring is done to copy data from old disks to new disks  and release the old disks to SAN. I trust you have taken the necessary steps to an ignite, health check, backup, Change Request, etc. Tested on HPUX 11.23

We start by adding new mirror disks to the server

Create a PV for all newly added LUN. We use the raw character of a disk whenever we're adding new a disk to a server. I highlighted this in yellow

super22:/etc# pvcreate /dev/rdsk/c44t10d2
Physical volume "/dev/rdsk/c44t10d2" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c45t10d2
Physical volume "/dev/rdsk/c45t10d2" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c44t10d3
Physical volume "/dev/rdsk/c44t10d3" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c45t10d3
Physical volume "/dev/rdsk/c45t10d3" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c44t10d4
Physical volume "/dev/rdsk/c44t10d4" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c45t10d4
Physical volume "/dev/rdsk/c45t10d4" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c44t10d5
Physical volume "/dev/rdsk/c44t10d5" has been successfully created.

super22:/etc# pvcreate /dev/rdsk/c45t10d5
Physical volume "/dev/rdsk/c45t10d5" has been successfully created.
super22:/etc# 

1) Append in the /etc/lvmpvg and add the new mirror disks. Note that I have not used the raw device character when adding disk to lvmpvg

PVG     PVG1
/dev/dsk/c44t10d2
/dev/dsk/c44t10d3
/dev/dsk/c44t10d4

/dev/dsk/c44t10d5

2) We then extend VG vgSPAN with the new mirror disks

super22:/etc# vgextend vgSPAN /dev/dsk/c44t10d2 /dev/dsk/c44t10d3 /dev/dsk/c44t10d4 /dev/dsk/c44t10d5
Current path "/dev/dsk/c35t8d2" is an alternate link, skip.
Current path "/dev/dsk/c33t8d3" is an alternate link, skip.
Current path "/dev/dsk/c35t8d4" is an alternate link, skip.
Current path "/dev/dsk/c33t8d5" is an alternate link, skip.
Current path "/dev/dsk/c35t8d6" is an alternate link, skip.
Current path "/dev/dsk/c33t8d7" is an alternate link, skip.
Current path "/dev/dsk/c45t9d4" is an alternate link, skip.
Current path "/dev/dsk/c45t9d5" is an alternate link, skip.
Current path "/dev/dsk/c45t9d6" is an alternate link, skip.
Current path "/dev/dsk/c45t9d7" is an alternate link, skip.
Current path "/dev/dsk/c45t10d0" is an alternate link, skip.
Current path "/dev/dsk/c45t10d1" is an alternate link, skip.
Volume group "vgSPAN" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

super22:/etc# vgextend vgSPAN /dev/dsk/c45t10d2 /dev/dsk/c45t10d3 /dev/dsk/c45t10d4 /dev/dsk/c45t10d5
Current path "/dev/dsk/c35t8d2" is an alternate link, skip.
Current path "/dev/dsk/c33t8d3" is an alternate link, skip.
Current path "/dev/dsk/c35t8d4" is an alternate link, skip.
Current path "/dev/dsk/c33t8d5" is an alternate link, skip.
Current path "/dev/dsk/c35t8d6" is an alternate link, skip.
Current path "/dev/dsk/c33t8d7" is an alternate link, skip.
Current path "/dev/dsk/c45t9d4" is an alternate link, skip.
Current path "/dev/dsk/c45t9d5" is an alternate link, skip.
Current path "/dev/dsk/c45t9d6" is an alternate link, skip.
Current path "/dev/dsk/c45t9d7" is an alternate link, skip.
Current path "/dev/dsk/c45t10d0" is an alternate link, skip.
Current path "/dev/dsk/c45t10d1" is an alternate link, skip.
Volume group "vgSPAN" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

super22:/etc# 

View the VG while you're at it

super22:/etc# vgdisplay vgSPAN
--- Volume groups ---
VG Name                     /dev/vgSPAN
VG Write Access             read/write     
VG Status                   available                 
Max LV                      255    
Cur LV                      5      
Open LV                     5      
Max PV                      16     
Cur PV                      16     
Act PV                      16     
Max PE per PV               16000        
VGDA                        32  
PE Size (Mbytes)            32              
Total PE                    34160   
Alloc PE                    14066   
Free PE                     20094   
Total PVG                   2        
Total Spare PVs             0              
Total Spare PVs in use      0    


And here is the list of old disks at the LVs of VG vgSPAN. 

Old DISKS at LVs of VgSPAN:

LV1

--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol1
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0   ----------->Not mirrored yet          
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            4224            
Current LE                  132       
Allocated PE                132         
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   22        22        
   /dev/dsk/c35t8d3   22        22        
   /dev/dsk/c33t8d4   22        22        
   /dev/dsk/c35t8d5   22        22        
   /dev/dsk/c33t8d6   22        22        
   /dev/dsk/c35t8d7   22        22  

####################################################################

LV2:

super22:/etc# lvdisplay -v /dev/vgSPAN/lvol2 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol2
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0 ---------->Not mirrored yet            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            4224            
Current LE                  132       
Allocated PE                132         
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   22        22        
   /dev/dsk/c35t8d3   22        22        
   /dev/dsk/c33t8d4   22        22        
   /dev/dsk/c35t8d5   22        22        
   /dev/dsk/c33t8d6   22        22        
   /dev/dsk/c35t8d7   22        22  

####################################################################

LV3:

super22:/etc# lvdisplay -v /dev/vgSPAN/lvol3 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol3
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0 --------->Not mirrored yet            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            71680           
Current LE                  2240      
Allocated PE                2240        
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   214       214       
   /dev/dsk/c35t8d3   214       214       
   /dev/dsk/c33t8d4   214       214       
   /dev/dsk/c35t8d5   214       214       
   /dev/dsk/c33t8d6   214       214       
   /dev/dsk/c35t8d7   214       214       
   /dev/dsk/c44t9d4   160       160       
   /dev/dsk/c44t9d5   160       160       
   /dev/dsk/c44t9d6   159       159       
   /dev/dsk/c44t9d7   159       159       
   /dev/dsk/c44t10d0  159       159       
   /dev/dsk/c44t10d1  159       159 

####################################################################

LV4:

super22:/etc# lvdisplay -v /dev/vgSPAN/lvol4 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol4
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0  ---------->Not mirrored yet            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            368640          
Current LE                  11520     
Allocated PE                11520       
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   694       694       
   /dev/dsk/c35t8d3   694       694       
   /dev/dsk/c33t8d4   694       694       
   /dev/dsk/c35t8d5   694       694       
   /dev/dsk/c33t8d6   694       694       
   /dev/dsk/c35t8d7   694       694       
   /dev/dsk/c44t9d4   1226      1226      
   /dev/dsk/c44t9d5   1226      1226      
   /dev/dsk/c44t9d6   1226      1226      
   /dev/dsk/c44t9d7   1226      1226      
   /dev/dsk/c44t10d0  1226      1226      

   /dev/dsk/c44t10d1  1226      1226      

####################################################################

LV5:

super22:/etc# lvdisplay -v /dev/vgSPAN/lvol5 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol5
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               0 ---------->Not mirrored yet            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            1344            
Current LE                  42        
Allocated PE                42          
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   7         7         
   /dev/dsk/c35t8d3   7         7         
   /dev/dsk/c33t8d4   7         7         
   /dev/dsk/c35t8d5   7         7         
   /dev/dsk/c33t8d6   7         7         
   /dev/dsk/c35t8d7   7         7         


####################################################################

3. Mirror the LVs

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol1 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol1" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
super22:/root#

(Make LV status is syncd & mirror copy is 1)

super22:/root# lvdisplay /dev/vgSPAN/lvol1 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               1 --------------->Mirrored now      
spthpn22:/root#

super22:/root# lvdisplay -v /dev/vgSPAN/lvol1 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol1
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            4224            
Current LE                  132       
Allocated PE                264         
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/rdsk/c33t8d2   22        22        
   /dev/dsk/c35t8d3   22        22        
   /dev/dsk/c33t8d4   22        22        
   /dev/dsk/c35t8d5   22        22        
   /dev/dsk/c33t8d6   22        22        
   /dev/dsk/c35t8d7   22        22        
   /dev/dsk/c44t10d2  33        33 --------->new disks mirrored      
   /dev/dsk/c44t10d3  33        33 --------->new disks mirrored        
   /dev/dsk/c44t10d4  33        33 --------->new disks mirrored        
   /dev/dsk/c44t10d5  33        33 --------->new disks mirrored        


####################################################################

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol2 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol2" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
spthpn22:/root# 

super22:/root# lvdisplay /dev/vgSPAN/lvol2 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               1 --------------->Mirrored now             
spthpn22:/root# 

super22:/root# lvdisplay -v /dev/vgSPAN/lvol2 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol2
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            4224            
Current LE                  132       
Allocated PE                264         
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   22        22        
   /dev/dsk/c35t8d3   22        22        
   /dev/dsk/c33t8d4   22        22        
   /dev/dsk/c35t8d5   22        22        
   /dev/dsk/c33t8d6   22        22        
   /dev/dsk/c35t8d7   22        22        
   /dev/dsk/c44t10d2  33        33 --------->new disks mirrored       
   /dev/dsk/c44t10d3  33        33 --------->new disks mirrored         
   /dev/dsk/c44t10d4  33        33 --------->new disks mirrored         
   /dev/dsk/c44t10d5  33        33 --------->new disks mirrored   


####################################################################

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol3 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol3" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf 

super22:/root# lvdisplay /dev/vgSPAN/lvol3 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               1 --------------->Mirrored now            
spthpn22:/root# 

super22:/root# lvdisplay -v /dev/vgSPAN/lvol3 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol3
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            71680           
Current LE                  2240      
Allocated PE                4480        
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   214       214       
   /dev/dsk/c35t8d3   214       214       
   /dev/dsk/c33t8d4   214       214       
   /dev/dsk/c35t8d5   214       214       
   /dev/dsk/c33t8d6   214       214       
   /dev/dsk/c35t8d7   214       214       
   /dev/dsk/c44t9d4   160       160       
   /dev/dsk/c44t9d5   160       160       
   /dev/dsk/c44t9d6   159       159       
   /dev/dsk/c44t9d7   159       159       
   /dev/dsk/c44t10d0  159       159         
   /dev/dsk/c44t10d1  159       159         
   /dev/dsk/c44t10d2  560       560 --------->new disks mirrored         
   /dev/dsk/c44t10d3  560       560 --------->new disks mirrored         
   /dev/dsk/c44t10d4  560       560 --------->new disks mirrored         
   /dev/dsk/c44t10d5  560       560 --------->new disks mirrored   


####################################################################

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol4 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol4" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
super22:/root# 

super22:/root# lvdisplay -v /dev/vgSPAN/lvol4 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol4
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            ------------->Mirrored now 
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            368640          
Current LE                  11520     
Allocated PE                23040       
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   694       694       
   /dev/dsk/c35t8d3   694       694       
   /dev/dsk/c33t8d4   694       694       
   /dev/dsk/c35t8d5   694       694       
   /dev/dsk/c33t8d6   694       694       
   /dev/dsk/c35t8d7   694       694       
   /dev/dsk/c44t9d4   1226      1226      
   /dev/dsk/c44t9d5   1226      1226      
   /dev/dsk/c44t9d6   1226      1226      
   /dev/dsk/c44t9d7   1226      1226      
   /dev/dsk/c44t10d0  1226      1226      
   /dev/dsk/c44t10d1  1226      1226      
   /dev/dsk/c44t10d2  2880      2880 --------->new disks mirrored      
   /dev/dsk/c44t10d3  2880      2880 --------->new disks mirrored       
   /dev/dsk/c44t10d4  2880      2880 --------->new disks mirrored       
   /dev/dsk/c44t10d5  2880      2880 --------->new disks mirrored  


####################################################################

super22:/root# lvextend -m 1 /dev/vgSPAN/lvol5 PVG1
The newly allocated mirrors are now being synchronized. This operation will 
take some time. Please wait ....
Logical volume "/dev/vgSPAN/lvol5" has been successfully extended.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

super22:/root# lvdisplay /dev/vgSPAN/lvol5 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               1 --------------->Mirrored now             
spthpn22:/root# 

super22:/root# lvdisplay -v /dev/vgSPAN/lvol5 | more
--- Logical volumes ---
LV Name                     /dev/vgSPAN/lvol5
VG Name                     /dev/vgSPAN
LV Permission               read/write   
LV Status                   available/syncd           
Mirror copies               1            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            1344            
Current LE                  42        
Allocated PE                84          
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   NONE         
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name            LE on PV  PE on PV  
   /dev/dsk/c33t8d2   7         7         
   /dev/dsk/c35t8d3   7         7         
   /dev/dsk/c33t8d4   7         7         
   /dev/dsk/c35t8d5   7         7         
   /dev/dsk/c33t8d6   7         7         
   /dev/dsk/c35t8d7   7         7         
   /dev/dsk/c44t10d2  11        11 --------->new disks mirrored        
   /dev/dsk/c44t10d3  11        11 --------->new disks mirrored        
   /dev/dsk/c44t10d4  10        10 --------->new disks mirrored        
   /dev/dsk/c44t10d5  10        10 -------->new disks mirrored 



####################################################################

While mirroring is going on, to check the status of mirror-because it takes time, run this command

# lvdisplay -v /dev/vgSPAN/lvolx |grep stale|wc -l

The number of stale PE will decline and when it hits zero that's when you know mirroring has completed 

5. Once all the 5 LVs have been mirrored, you remove/un-mirror the old LUNs from all the five LVs of VG vgSPAN

# lvreduce -m 0 /dev/vgSPAN/lvol1 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7

super22:/root# lvreduce -m 0 /dev/vgSPAN/lvol1 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 
Logical volume "/dev/vgSPAN/lvol1" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

# lvreduce -m 0 /dev/vgSPAN/lvol2 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7

super22:/root# /c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7                                     <
Logical volume "/dev/vgSPAN/lvol2" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf


# lvreduce -m 0 /dev/vgSPAN/lvol3 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7 /dev/dsk/c44t9d4 /dev/dsk/c44t9d5 /dev/dsk/c44t9d6 /dev/dsk/c44t9d7 /dev/dsk/c44t10d0 /dev/dsk/c44t10d1

super22:/root# v/dsk/c44t9d6 /dev/dsk/c44t9d7 /dev/dsk/c44t10d0 /dev/dsk/c44t10d1                                               <
Logical volume "/dev/vgSPAN/lvol3" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
super22:/root# 


# lvreduce -m 0 /dev/vgSPAN/lvol4 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7 /dev/dsk/c44t9d4 /dev/dsk/c44t9d5 /dev/dsk/c44t9d6 /dev/dsk/c44t9d7 /dev/dsk/c44t10d0 /dev/dsk/c44t10d1

super22:/root# v/dsk/c44t9d6 /dev/dsk/c44t9d7 /dev/dsk/c44t10d0 /dev/dsk/c44t10d1                                               <
Logical volume "/dev/vgSPAN/lvol4" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf
super22:/root# 

# lvreduce -m 0 /dev/vgSPAN/lvol5 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7

super22:/root# /c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7                                     <
Logical volume "/dev/vgSPAN/lvol5" has been successfully reduced.
Volume Group configuration for /dev/vgSPAN has been saved in /etc/lvmconf/vgSPAN.conf

super22:/root# 

Running lvreduce command can be tricky, and an error is bound to happen if you don't remove all the disks together in a single command line. If that happens don't panic. There's always a way to reverse engineer a bit

Say for an example I was suppose to remove 6 disks but I only removed 4, so I have to mirror back these disks into the LV it was removed from, and THEN UN-MIRROR them together

5.1. Identify 4 disks not used by lvol1 of VG vgSPAN:

/dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5

5.2. Mirror the 4 disks to LVOL1 back:

lvextend -m 1 /dev/vgSPAN/lvol1 /dev/dsk/c33t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d5

5.3. Check the mirror sync status:

lvdisplay /dev/vgSPAN/lvol1 |egrep "LV Status|Mirror copies"


5.4. Un-mirror all 6 disks together in a single command line:


lvreduce -m 0 /dev/vgSPAN/lvol1 /dev/dsk/c33t8d6 /dev/dsk/c35t8d7 /dev/dsk/c44t10d2 /dev/dsk/c44t10d3 /dev/dsk/c44t10d4  /dev/dsk/c44t10d5

####################################################################

6. Make sure all the LVs' mirror copy is 0

super22:/root# lvdisplay /dev/vgSPAN/lvol1 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0
super22:/root# lvdisplay /dev/vgSPAN/lvol2 | egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0            
spthpn22:/root#

super22:/root# lvdisplay /dev/vgSPAN/lvol3 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0            
spthpn22:/root# 

super22:/root#  lvdisplay /dev/vgSPAN/lvol4 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0            
spthpn22:/root# 

super22:/root#  lvdisplay /dev/vgSPAN/lvol5 |egrep "LV Status|Mirror copies"
LV Status                   available/syncd           
Mirror copies               0            
spthpn22:/root# 

####################################################################

7. Remove/release old LUN from vgSPAN

# vgreduce vgSPAN /dev/dsk/c33t8d2 /dev/dsk/c35t8d2 /dev/dsk/c35t8d3 /dev/dsk/c33t8d3 /dev/dsk/c33t8d4 /dev/dsk/c35t8d4 /dev/dsk/c35t8d5 /dev/dsk/c33t8d5 /dev/dsk/c33t8d6 /dev/dsk/c35t8d6  /dev/dsk/c35t8d7 /dev/dsk/c33t8d7   

        

# vgreduce vgSPAN /dev/dsk/c44t9d4 /dev/dsk/c45t9d4 /dev/dsk/c44t9d5 /dev/dsk/c45t9d5 /dev/dsk/c44t9d6 /dev/dsk/c45t9d6 /dev/dsk/c44t9d7 /dev/dsk/c45t9d7 /dev/dsk/c44t10d0 /dev/dsk/c45t10d0 /dev/dsk/c44t10d1 /dev/dsk/c45t10d1

8. Remove the old disks entries from /etc/lvmpvg 

PVG     PVG0
/dev/dsk/c33t8d2
/dev/dsk/c35t8d3
/dev/dsk/c33t8d4
/dev/dsk/c35t8d5
/dev/dsk/c33t8d6
/dev/dsk/c35t8d7
/dev/dsk/c44t9d4
/dev/dsk/c44t9d5
/dev/dsk/c44t9d6
/dev/dsk/c44t9d7
/dev/dsk/c44t10d0
/dev/dsk/c44t10d1

9.  Issue "rmsf /dev/dsk/cXtXdX" on following disks to completely remove them from the server

rmsf /dev/dsk/c33t8d2
rmsf /dev/dsk/c35t8d2 
rmsf /dev/dsk/c35t8d3
rmsf /dev/dsk/c33t8d3 
rmsf /dev/dsk/c33t8d4
rmsf /dev/dsk/c35t8d4 
rmsf /dev/dsk/c35t8d5
rmsf /dev/dsk/c33t8d5 
rmsf /dev/dsk/c33t8d6
rmsf /dev/dsk/c35t8d6 
rmsf /dev/dsk/c35t8d7
rmsf /dev/dsk/c33t8d7 
rmsf /dev/dsk/c44t9d4
rmsf /dev/dsk/c45t9d4 
rmsf /dev/dsk/c44t9d5
rmsf /dev/dsk/c45t9d5 
rmsf /dev/dsk/c44t9d6
rmsf /dev/dsk/c45t9d6 
rmsf /dev/dsk/c44t9d7
rmsf /dev/dsk/c45t9d7 
rmsf /dev/dsk/c44t10d0
rmsf /dev/dsk/c45t10d0
rmsf /dev/dsk/c44t10d1
rmsf /dev/dsk/c45t10d1

10. Alternatively, you can use the commands rmsf -H H/W PATH instead of the rmsf /dev/dsk/XXX

11. Return the LUNs to SAN team

/dev/rdsk/c33t8d2            e1  08  42  CL1B  04:42  OPEN-V           00045468
/dev/rdsk/c35t8d2            d3  08  42  CL2B  04:42  OPEN-V           00045468
/dev/rdsk/c35t8d3            d3  08  43  CL2B  04:43  OPEN-V           00045468
/dev/rdsk/c33t8d3            e1  08  43  CL1B  04:43  OPEN-V           00045468
/dev/rdsk/c33t8d4            e1  08  44  CL1B  04:44  OPEN-V           00045468
/dev/rdsk/c35t8d4            d3  08  44  CL2B  04:44  OPEN-V           00045468
/dev/rdsk/c35t8d5            d3  08  45  CL2B  04:45  OPEN-V           00045468
/dev/rdsk/c33t8d5            e1  08  45  CL1B  04:45  OPEN-V           00045468
/dev/rdsk/c33t8d6            e1  08  46  CL1B  04:46  OPEN-V           00045468
/dev/rdsk/c35t8d6            d3  08  46  CL2B  04:46  OPEN-V           00045468
/dev/rdsk/c35t8d7            d3  08  47  CL2B  04:47  OPEN-V           00045468
/dev/rdsk/c33t8d7            e1  08  47  CL1B  04:47  OPEN-V           00045468
/dev/rdsk/c44t9d4            e1  09  cc  CL1B  06:09  OPEN-V           00045468
/dev/rdsk/c45t9d4            d3  09  cc  CL2B  06:09  OPEN-V           00045468
/dev/rdsk/c44t9d5            e1  09  cd  CL1B  06:0a  OPEN-V           00045468
/dev/rdsk/c45t9d5            d3  09  cd  CL2B  06:0a  OPEN-V           00045468
/dev/rdsk/c44t9d6            e1  09  ce  CL1B  06:0b  OPEN-V           00045468
/dev/rdsk/c45t9d6            d3  09  ce  CL2B  06:0b  OPEN-V           00045468
/dev/rdsk/c44t9d7            e1  09  cf  CL1B  06:0c  OPEN-V           00045468
/dev/rdsk/c45t9d7            d3  09  cf  CL2B  06:0c  OPEN-V           00045468
/dev/rdsk/c44t10d0           e1  0a  d0  CL1B  06:0d  OPEN-V           00045468
/dev/rdsk/c45t10d0           d3  0a  d0  CL2B  06:0d  OPEN-V           00045468
/dev/rdsk/c44t10d1           e1  0a  d1  CL1B  06:0e  OPEN-V           00045468
/dev/rdsk/c45t10d1           d3  0a  d1  CL2B  06:0e  OPEN-V           00045468

And we're done!