Tuesday, November 24, 2015

Linux : Create XFS file system

# pvcreate /dev/sdb
WARNING: dos signature detected on /dev/sdb at offset 510. Wipe it? [y/n]: y
  Wiping dos signature on /dev/sdb.
  Physical volume "/dev/sdb" successfully created

# vgcreate vg01 /dev/sdb
  Volume group "vg01" successfully created
# lvcreate -l +100%FREE -n lv01 vg01
  Logical volume "lv01" created.

# mkfs.xfs /dev/vg01/lv01
meta-data=/dev/vg01/lv01         isize=256    agcount=4, agsize=6553344 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=26213376, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=12799, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

# vi /etc/fstab
/dev/vg01/lv01          /data                   xfs     defaults        0 0

# mkdir /data
# mount /data
# df /data
Filesystem            1K-blocks  Used Available Use% Mounted on
/dev/mapper/vg01-lv01 104802308 32928 104769380   1% /data

Saturday, November 21, 2015

Friday, November 20, 2015

R : move selected column to the left

install.packages("dplyr")

library(dplyr)

# move Survived column to the left and keep everything else the same
train <- train2 %>% select(Survived, everything())

Linux : NIC interface is adding default router

Fedora 22

In dynamic network environments, where mobile hosts are managed by NetworkManager, gateway information is likely to be interface specific and is best left to be assigned by DHCP. In special cases where it is necessary to influence NetworkManager's selection of the exit interface to be used to reach a gateway, make use of the DEFROUTE=no command in the ifcfg files for those interfaces which do not lead to the default gateway.
Global default gateway configuration is stored in the /etc/sysconfig/network file. This file specifies gateway and host information for all network interfaces.

Python : Installing ipython-notebook with Miniconda

conda install ipython-notebook

start it:
ipython notebook


http://opentechschool.github.io/python-data-intro/core/notebook.html


Python : Installing pandas with Miniconda

http://pandas-docs.github.io/pandas-docs-travis/install.html

conda install pandas

Continuum Analytics conda repositories
http://repo.continuum.io/pkgs/index.html

Tuesday, November 17, 2015

OVM : use NFS version 3 by default on Oracle VM environment

To use NFS version 3 by default on Oracle VM environment, please log into Oracle VM servers:
- Please change the lines below in /etc/nfsmount.conf with "3":
[root@tz1 ~]# cat /etc/nfsmount.conf |grep -i "Defaultvers"
Defaultvers=3
[root@tz1 ~]# cat /etc/nfsmount.conf |grep -i "NFSvers"
Nfsvers=3

- Please change the line below in /etc/sysconfig/nfs with "yes":
[root@tz1 ~]# cat /etc/sysconfig/nfs |grep -i "MOUNTD_NFS_V3"
MOUNTD_NFS_V3="yes"

- Restart NFS service on OVS:
[root@tz1 ~]# service nfs restart

- Re-present the NFS share on OVM manager GUI.
- [root@tz1 ~]# mount -t nfs
xx.oracle.com:/mnt on /OVS/Repositories/0004fb00000300004b3974397e1a9d17 type nfs (rw,nosharecache,soft,fg,retry=1,nfsvers=3,addr=xx.xx.xx.xx.xx)

Note: Need to do the changes on all Oracle VM servers if you wants all NFS share to be mounted as version 3.

OVM 3.3.3

Friday, November 13, 2015

R : position in qplot

qplot(color,data=diamonds, fill=cut, position ="dodge")


"dodge" : Overlapping geoms are placed beside each other

"statck" : Overlapping geoms are placed one above the other

"identity" : No adjustment. Geoms are allowed to overlap

"fill" : The available space is divided proportionately between the overlapping geoms.

"jitter" : Random noise is added to the position of each geom

Thursday, November 12, 2015

Machine Learning : Tensor Flow

http://www.tensorflow.org/

https://github.com/tensorflow/tensorflow


Linux : Red Hat 7 commands and configuration files

1 System Locale and Keyboard Configuration

/etc/locale.conf

# cat /etc/locale.conf
LANG="en_US.UTF-8"

# localectl status
   System Locale: LANG=en_US.UTF-8
       VC Keymap: us
      X11 Layout: us

# localectl list-locales

# localectl set-locale LANG=locale

# localectl list-keymaps

# localectl set-keymap map

2 Configuring the Date and Time

A real-time clock (RTC), a hardware clock (on the system board)
A system clock, a software clock, maintained by the kernel

timedatectl

# timedatectl
      Local time: Thu 2015-11-12 14:17:33 EST
  Universal time: Thu 2015-11-12 19:17:33 UTC
        RTC time: Thu 2015-11-12 19:15:33
        Timezone: America/New_York (EST, -0500)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
      DST active: no
 Last DST change: DST ended at
                  Sun 2015-11-01 01:59:59 EDT
                  Sun 2015-11-01 01:00:00 EST
 Next DST change: DST begins (the clock jumps one hour forward) at
                  Sun 2016-03-13 01:59:59 EST
                  Sun 2016-03-13 03:00:00 EDT

If changes to the configuration of chrony or ntpd tools, enter the following command:
systemctl restart systemd-timedated.services

timedatectl set-time HH:MM:SS

This command updates both the system time and the hardware clock. The result is similar to using both date --set and hwclock --systohc commands.

 By default, the system is configured to use UTC. To configure your system to maintain the clock in the local time, run the timedatectl command with the set-local-rtc option as root:

timedatectl set-local-rtc boolean

To configure your system to maintain the clock in the local time, replace boolean with yes (or, alternatively, y, true, t, or 1). To configure the system to use UTC, replace boolean with no (or, alternatively, n, false, f, or 0). The default option is no.

 To change the current date, type the following at a shell prompt as root:

timedatectl set-time YYYY-MM-DD

Note that changing the date without specifying the current time results in setting the time to 00:00:00.

timedatectl set-time 'YYYY-MM-DD HH:MM:SS'

timedatectl list-timezones

To change the currently used time zone, type as root:

timedatectl set-timezone time_zone

To enable automatic synchronization of the system clock with a remote server, type:

~]# timedatectl set-ntp yes

 To display the current date and local time, type the following at a shell prompt:

~]$ date
Mon Sep 16 17:30:24 CEST 2013

To display the current date and time in UTC, type the following at a shell prompt:

~]$ date --utc
Mon Sep 16 15:30:34 UTC 2013

To customize the output of the date command, type:

~]$ date +"%Y-%m-%d %H:%M"
2013-09-16 17:30

 To change the current time, run the date command with the --set or -s option as root:

date --set HH:MM:SS

Replace HH with an hour, MM with a minute, and SS with a second, all typed in two-digit form.
By default, the date command sets the system clock to the local time. To set the system clock in UTC, run the command with the --utc or -u command line option:

date --set HH:MM:SS --utc

 To change the current date, run the date command with the --set or -s option as root:

date --set YYYY-MM-DD

Replace YYYY with a four-digit year, MM with a two-digit month, and DD with a two-digit day of the month.
Note that changing the date without specifying the current time results in setting the time to 00:00:00.

To change the current date to 2 June 2013 and keep the current time (11:26 p.m.), run the following command as root:

~]# date --set 2013-06-02 23:26:00

 hwclock is a utility for accessing the hardware clock, also referred to as the Real Time Clock (RTC). The hardware clock is independent of the operating system you use and works even when the machine is shut down. This utility is used for displaying the time from the hardware clock. hwclock also contains facilities for compensating for systematic drift in the hardware clock.
The hardware clock stores the values of: year, month, day, hour, minute, and second. It is not able to store the time standard, local time or Coordinated Universal Time (UTC), nor set the Daylight Saving Time (DST).
The hwclock utility saves its settings in the /etc/adjtime file, which is created with the first change you make, for example, when you set the time manually or synchronize the hardware clock with the system time.

 Running hwclock with no command line options as the root user returns the date and time in local time to standard output.

hwclock

 When you need to change the hardware clock date and time, you can do so by appending the --set and --date options along with your specification:

hwclock --set --date "dd mmm yyyy HH:MM"

 At the same time, you can also set the hardware clock to keep the time in either UTC or local time by adding the --utc or --localtime options, respectively. In this case, UTC or LOCAL is recorded in the /etc/adjtime file.

If you want to set the date and time to a specific value, for example, to "21:17, October 21, 2014", and keep the hardware clock in UTC, run the command as root in the following format:

~]# hwclock --set --date "21 Oct 2014 21:17" --utc

 You can synchronize the hardware clock and the current system time in both directions.

    Either you can set the hardware clock to the current system time by using this command:

    hwclock --systohc

    Note that if you use NTP, the hardware clock is automatically synchronized to the system clock every 11 minutes, and this command is useful only at boot time to get a reasonable initial system time.
    Or, you can set the system time from the hardware clock by using the following command:

    hwclock --hctosys

When you synchronize the hardware clock and the system time, you can also specify whether you want to keep the hardware clock in local time or UTC by adding the --utc or --localtime option. Similarly to using --set, UTC or LOCAL is recorded in the /etc/adjtime file.
The hwclock --systohc --utc command is functionally similar to timedatectl set-local-rtc false and the hwclock --systohc --local command is an alternative to timedatectl set-local-rtc true.

 To set the hardware clock to the current system time and keep the hardware clock in local time, run the following command as root:

~]# hwclock --systohc --localtime

To avoid problems with time zone and DST switching, it is recommended to keep the hardware clock in UTC

Monday, November 9, 2015

Linux : kernel:BUG: soft lockup

Encountered this problem in syslog

kernel:BUG: soft lockup - CPU#2 stuck for 22s! [process name:PID]

According to this link : http://askubuntu.com/questions/401736/what-does-cpu0-stuck-mean

The Linux kernel has a process which monitors each CPU on the system.
There are special interrupt(s) in the kernel. This interrupt(s) function calls a soft-lockup counter, it will compare the current time stamp with the specific kernel CPU data structure time information. If it looks like the current time stamp is greater than the defined threshold (in seconds) later as compared to the stored time stamp, it is assumed that the monitoring process or watchdog thread(s) have not executed in a respectable amount of time.
Why or how can a CPU soft lock occur? How can a CPU get locked if the kernel is carefully scheduling CPU access? Basically any poorly written code that loops a lot or infinitely, would own a CPU and get some priority. It can be a programming problem or 3rd party software.
Locking issues in drivers. Even kernel bugs in important drivers or the scheduler. A scheduler could tell schedule a driver routine to run and if that driver has problems and doesn’t check on it, that driver routine could own or hog that CPU for a longtime. By definition as described above, the watchdog would catch this and issue a soft lockup alert.

Linux : rsync

DRYRUN="--dry-run"
#DRYRUN=""

/usr/bin/rsync -v -e ssh $DRYRUN -a -H --exclude archive --exclude "lost+found" --exclude config --exclude log --rsync-path=/usr/bin/rsync --delete /source_DIR DESTINATION_SERVER:/DEST_DIR

OVM : MTU downgrade problem

https://oss.oracle.com/pipermail/oraclevm-errata/2015-February/000275.html

[4.3.0-55.el6.1] - Before connecting the emulated network interface (vif.x.y-emu) to a bridge, change the emu MTU to equal the MTU of the bridge to prevent the bridge from downgrading its own MTU to equal the emu MTU.

Thursday, November 5, 2015

R : install ggplot2 package on Windows 8

Install Rtools for Windows : https://cran.r-project.org/bin/windows/Rtools/

Without RTools, we cannot compile the package.

In the R console , run :

install.packages("ggplot2", type="source",dependencies = TRUE)

Wednesday, November 4, 2015

R : stringsAsFactors

https://www.zoology.ubc.ca/~schluter/R/data/

stringsAsFactors = FALSE tells R to keep character variables as they are rather than convert to factors

OpenStack : Nova Error : NoValidHost

Problem:

On Controller node , File /var/log/nova/nova-conductor.log

2015-10-23 18:20:30.836 1572 ERROR nova.scheduler.utils [req-20b92c63-df02-4c16-941e-5b7c0b68aa96 b476579221d74f1eb0474fb5d970dead eec1d66ee98a46458c98f2c84607c229 - - -] [instance: d39eb1ab-14f8-4bc7-8029-3c5fb92cdf95] Error from last host: fedora-68 (node fedora-68): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2235, in _do_build_and_run_instance\n    filter_properties)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2381, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance d39eb1ab-14f8-4bc7-8029-3c5fb92cdf95 was re-scheduled: Connection to glance host IP:9292 failed: Error finding address for http://IP:9292/v1/images/3f65b449-a945-4247-bcc2-e33d94a07cb6: ('Connection aborted.', error(113, 'EHOSTUNREACH'))\n"]
2015-10-23 18:20:32.368 1572 WARNING nova.scheduler.utils [req-20b92c63-df02-4c16-941e-5b7c0b68aa96 b476579221d74f1eb0474fb5d970dead eec1d66ee98a46458c98f2c84607c229 - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available.
Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner
    return func(*args, **kwargs)

  File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 86, in select_destinations
    filter_properties)

  File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 80, in select_destinations
    raise exception.NoValidHost(reason=reason)

NoValidHost: No valid host was found. There are not enough hosts available.

Solution : Check firewall for Controller node

Big Data : Spark

http://www.computing.co.uk/ctg/analysis/2432898/what-is-spark-six-reasons-why-cios-should-find-out-and-one-why-they-shouldnt

Tuesday, November 3, 2015

Oracle and VMware Support


VMware: http://www.vmware.com/support/policies/oracle-support.html


Oracle: https://support.oracle.com/epmos/faces/DocumentDisplay?id=249212.1

 Oracle has not certified any of its products on VMware virtualized environments. Oracle Support will assist customers running Oracle products on VMware in the following manner: Oracle will only provide support for issues that either are known to occur on the native OS, or can be demonstrated not to be as a result of running on VMware. If a problem is a known Oracle issue, Oracle support will recommend the appropriate solution on the native OS. If that solution does not work in the VMware virtualized environment, the customer will be referred to VMware for support. When the customer can demonstrate that the Oracle solution does not work when running on the native OS, Oracle will resume support, including logging a bug with Oracle Development for investigation if required. If the problem is determined not to be a known Oracle issue, we will refer the customer to VMware for support. When the customer can demonstrate that the issue occurs when running on the native OS, Oracle will resume support, including logging a bug with Oracle Development for investigation if required. NOTE: Oracle has not certified any of its products on VMware. For Oracle RAC, Oracle will only accept Service Requests as described in this note on Oracle RAC 11.2.0.2 and later releases.