Keep the first match and remove the rest match from file
sed 'PATTERN\s*$/d' -i path_to_file
Tuesday, August 8, 2017
Wednesday, August 2, 2017
Python regex
#!/usr/bin/python
import re
regexPattern = re.compile('^(?!\\/usr\\b|\\/var)(\\/[\\w]+)+\\/?$')
inputString = [ "/", "/usr", "/var", "/test" ]
for v in inputString:
m = re.match(regexPattern, v)
print("")
print("Test String " + v )
if m:
print("Full Match" + m.group())
print("Pass")
else:
print("No Match")
Monday, July 24, 2017
Ansible pull
stdbuf -o0 -e0 ansible-pull \
-U URL of the playbook repository
-C branch/tag/commit
-e "key=value"
--purge 2>&1 | tee /path/to/ansible.log
Wednesday, July 19, 2017
Troubleshoot NFS
On Client:
tcpdump -i eth0 -s0 -w /tmp/tcpdump.pcap host example.com
Read the output
tcpdump -s0 -r /tmp/tcpdump.pcap
tcpdump -s0 dust port nfs -r /tmp/tcpdump.pcap
tcpdump -i eth0 -s0 -w /tmp/tcpdump.pcap host example.com
Read the output
tcpdump -s0 -r /tmp/tcpdump.pcap
tcpdump -s0 dust port nfs -r /tmp/tcpdump.pcap
Friday, July 14, 2017
Tensorflow multiple GPU
http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
CUDA_VISIBLE_DEVICES=0,1,2
https://www.tensorflow.org/tutorials/using_gpu
CUDA_VISIBLE_DEVICES=0,1,2
https://www.tensorflow.org/tutorials/using_gpu
Wednesday, July 12, 2017
Shell: Snippet
Read argument
while getopts a:b OPTIONS 2> /dev/null
do
case $(OPTIONS) in
a)
VAR1=${OPTARG}
;;
b)
VAR2=1
;;
*)
echo "Invalid"
usage
exit 1
;;
esac
done
logcmd ()
{
echo "" >> ${LOGFILE}
echo "Running: $@" >> ${LOGFILE}
$@ 2>&1 | tee -a ${LOGFILE}
return ${PIPESTATUS}
}
logcmd ()
{
echo "" >> ${LOGFILE}
echo "Running: $@" >> ${LOGFILE}
$@ 2>&1 | tee -a ${LOGFILE}
return ${PIPESTATUS}
}
Monday, May 1, 2017
NTP Config
For log file and time drift tolerance
/etc/ntp.conf
tinker step 0.05 (0.05 = 50 milliseconds)
logconfig=allall
logfile /var/log/ntpd.log
http://www.finra.org/industry/notices/16-23
/etc/ntp.conf
tinker step 0.05 (0.05 = 50 milliseconds)
logconfig=allall
logfile /var/log/ntpd.log
http://www.finra.org/industry/notices/16-23
PKI
PKI (Public Key Infrastructure) is a structured standards-based technology that is utilized by corporate intranets are well as internet-based eBusiness applications.
PKI is to authenticate users or machines and to encrypt data transmission between two entities.
The component of PKI:
* Certificate Authority(CA)
* Public/Private Key Pair
* Digital Certificate
* Certificate Trust Chain/Certificate Hierarchy
* Key Management
A CA is the trusted authority that issues digital certificates to individuals or to servers or websites to allow them to prove and attest to their identity.
Data encrypted with the public key can only be decrypted by the private key, and data signed by the private key can only be validated using the public key.
Digital Certificate is the electronic commerce worlds version of a password or driver's license.
The most widely accepted format for Digital Certificates is defined by the X.509 international standard. Basic attributes of the X.509 standard are the owner's name and public key, expiration date of the public key, issuing Certificate Authority's name and its digital signature and the serial and version number of the Digital Certificate.
Alice
Sign email using private key
Encrypt email using Bob's public key
Bob
Encrypt email using Private key
Validate signature using Alice's public key
Certificate Trust chain:
Root CA
Intermediate/Subordinate CA
Issuing CA
Key Management: Private keys must be securely generated and stored.
Certificate Encoding Formats
Base64
PEM (privacy enhanced mail) format is the most common format that Certificate Authorities certificates in. PEM certificates usually have extensions such as .pem , .crt, .cer and .key. PEM format is Base64 ASCII format.
DER (Distinguished Encoding Rules) format supports storage of a single certificate. The DER format is a binary form of a certificate.
SSL
Secure Socket Layer technology is the global standard security technology for establishing an encrypted link for secure data exchange between clients and web servers.
SSL 3.0 served as the basis for TLS 1.0.
The Internet Engineering Task Force (IETF) created TLS (Transport Layer Security) as the successor to SSL.
SSL/TLS communication:
1 Client visits secure web site https:// (ClientHello)
2 Server asserts its site identity by sending back its own SSL certificate's public key to the client's browser. (Server Hello)
3 Client verifies the server's identity. Performs certificate chain validation. (trusted CA, not expired, not revoked)
4 Server sends a request to the client to send its client certificate.
5 Client selects and sends an appropriate certificate to the server
6 Server verifies client authenticity
7 Client : When mutual authentication is complete and both client and server are assured, the client then generates a key called "session key" or "pre-master key", encrypts it using the server's public key, send the pre-master key to the server.
8 Server uses its private key to decrypt the pre-master key and generate master key for data exchange.
9 Client and server use the negotiated master key to encrypt and decrypt the data.
Use private key size of 2048 to ensure strong authentication.
Comply with Payment Card Industry (PCI DSS) standards.
PKI is to authenticate users or machines and to encrypt data transmission between two entities.
The component of PKI:
* Certificate Authority(CA)
* Public/Private Key Pair
* Digital Certificate
* Certificate Trust Chain/Certificate Hierarchy
* Key Management
A CA is the trusted authority that issues digital certificates to individuals or to servers or websites to allow them to prove and attest to their identity.
Data encrypted with the public key can only be decrypted by the private key, and data signed by the private key can only be validated using the public key.
Digital Certificate is the electronic commerce worlds version of a password or driver's license.
The most widely accepted format for Digital Certificates is defined by the X.509 international standard. Basic attributes of the X.509 standard are the owner's name and public key, expiration date of the public key, issuing Certificate Authority's name and its digital signature and the serial and version number of the Digital Certificate.
Alice
Sign email using private key
Encrypt email using Bob's public key
Bob
Encrypt email using Private key
Validate signature using Alice's public key
Certificate Trust chain:
Root CA
Intermediate/Subordinate CA
Issuing CA
Key Management: Private keys must be securely generated and stored.
Certificate Encoding Formats
Base64
PEM (privacy enhanced mail) format is the most common format that Certificate Authorities certificates in. PEM certificates usually have extensions such as .pem , .crt, .cer and .key. PEM format is Base64 ASCII format.
DER (Distinguished Encoding Rules) format supports storage of a single certificate. The DER format is a binary form of a certificate.
SSL
Secure Socket Layer technology is the global standard security technology for establishing an encrypted link for secure data exchange between clients and web servers.
SSL 3.0 served as the basis for TLS 1.0.
The Internet Engineering Task Force (IETF) created TLS (Transport Layer Security) as the successor to SSL.
SSL/TLS communication:
1 Client visits secure web site https:// (ClientHello)
2 Server asserts its site identity by sending back its own SSL certificate's public key to the client's browser. (Server Hello)
3 Client verifies the server's identity. Performs certificate chain validation. (trusted CA, not expired, not revoked)
4 Server sends a request to the client to send its client certificate.
5 Client selects and sends an appropriate certificate to the server
6 Server verifies client authenticity
7 Client : When mutual authentication is complete and both client and server are assured, the client then generates a key called "session key" or "pre-master key", encrypts it using the server's public key, send the pre-master key to the server.
8 Server uses its private key to decrypt the pre-master key and generate master key for data exchange.
9 Client and server use the negotiated master key to encrypt and decrypt the data.
Use private key size of 2048 to ensure strong authentication.
Comply with Payment Card Industry (PCI DSS) standards.
Wednesday, April 26, 2017
Troubleshooting
. When did this issue begin occurring? - Does this time and date coincide with any other events? . How often does it occur? - Is is continuous or off and on? - If off and on, is there any pattern to the events? - Are there any other events that coincide when the issue occurs? Like backups or peak business hours? . What is the time stamp of the last time this occurred? - Does this time and date coincide with any other events? . Is it the entire system or is it a particular subsystem/application? For example, is the slowness at the console as well as over ssh or is it specific to an action, is it specific to an application, etc? . Does this same performance impact occur when all 3rd party or only certain 3rd party applications are disabled? . Where and how is the performance impact being observed? . Can a complete description of what the performance is supposed be provided? In particular how the expected performance was measured, what the results were, etc . What is the present status of the system? Is it working normally? or is it experiencing any issues? . What type of workload is this system responsible for ordinarily? I.E. Does it host a database, fileserver, or other application? . Were there any power or networking outages reported near the time that this issue occurred? . Do you know of any CPU and/or memory intensive process that could have been running on the system at the time of the issue? . Were there any messages on the console at the time of the issue? If so, can they be provided to the case in their entirety? . Is this system part of a cluster? . Does the issue cause the system to have a vmcore generated? . Were there any recent changes to the hardware/firmware/software recently? If so, when and what was changed?
Please clarify the issue "servers running slow" - Is some process not completing on time? - Are commands issued to the system taking a long time to complete? What benchmark or comparison do you have to indicate that this is a problem? When did this issue begin occurring? Were any hardware or software changes or upgrades made before this issue began? Has the workload on the system changed at all? Is the issue persistent, repeatable, or random? Do you have any other data that may help in our investigation, including screenshots, comparisons indicating the performance loss, etc.
Wednesday, April 19, 2017
Ansible : snippet
- name: add search path to /etc/resolv.conf
lineinfile:
dest: "/etc/resolv.conf"
backrefs: True
state: present
insertbefore: ^nameserver
regexp: '(^\s*search(\s+(?!\b{{ item }}\b)[\w\.]+)*)\s*$'
line: '\1 {{ item }}'
with_items: "{{ searchpath }}"
About regex_replace
var1: ""{{ var2 | regex_replace ('ABC-(\\d.\\d+)', '\\1) }}"
ansible-vault encrypt_string password123 --ask-vault-pass
- name: revert to a snapshot
vmware_guest_snapshot:
datacenter:
hostname: this is vcenter host name
username:
password:
name: this is vm name
state: revert
snapshot_name:
delegate_to: localhost
- name: power on vm
vmware_guest:
datacenter:
hostname: this is vcenter host name
username:
password:
name: this is vm name
state: poweredon
delegate_to: localhost
- name: Wait for host to come back
local_action:
module: wait_for
host="{{ inventory_hostname }}"
port=22
delay=30
timeout=300
lineinfile:
dest: "/etc/resolv.conf"
backrefs: True
state: present
insertbefore: ^nameserver
regexp: '(^\s*search(\s+(?!\b{{ item }}\b)[\w\.]+)*)\s*$'
line: '\1 {{ item }}'
with_items: "{{ searchpath }}"
About regex_replace
var1: ""{{ var2 | regex_replace ('ABC-(\\d.\\d+)', '\\1) }}"
ansible-vault encrypt_string password123 --ask-vault-pass
- name: revert to a snapshot
vmware_guest_snapshot:
datacenter:
hostname: this is vcenter host name
username:
password:
name: this is vm name
state: revert
snapshot_name:
delegate_to: localhost
- name: power on vm
vmware_guest:
datacenter:
hostname: this is vcenter host name
username:
password:
name: this is vm name
state: poweredon
delegate_to: localhost
- name: Wait for host to come back
local_action:
module: wait_for
host="{{ inventory_hostname }}"
port=22
delay=30
timeout=300
Openshift
Openshift 3.5 Documentation: https://access.redhat.com/documentation/en/openshift-container-platform/?version=3.5
Release notes: https://access.redhat.com/documentation/en-us/openshift_container_platform/3.5/html-single/release_notes/#release-notes-ocp-3-5-release-notes
Upgrading: https://access.redhat.com/documentation/en-us/openshift_container_platform/3.5/html-single/installation_and_configuration/#install-config-upgrading-index
Scaling and Performance: https://access.redhat.com/documentation/en-us/openshift_container_platform/3.5/html-single/scaling_and_performance_guide/
Release notes: https://access.redhat.com/documentation/en-us/openshift_container_platform/3.5/html-single/release_notes/#release-notes-ocp-3-5-release-notes
Upgrading: https://access.redhat.com/documentation/en-us/openshift_container_platform/3.5/html-single/installation_and_configuration/#install-config-upgrading-index
Scaling and Performance: https://access.redhat.com/documentation/en-us/openshift_container_platform/3.5/html-single/scaling_and_performance_guide/
Wednesday, April 12, 2017
Kickstart
PXE doc
http://www.pix.net/software/pxeboot/archive/pxespec.pdf
https://docs.oracle.com/cd/E24628_01/em.121/e27046/appdx_pxeboot.htm#EMLCM12198
Kickstart sequence of events
http://www.linux-mag.com/id/6747/
http://www.pix.net/software/pxeboot/archive/pxespec.pdf
Understanding PXE Booting and Kickstart Technology
PXE works with Network Interface Card (NIC) of the system by making it function like a boot device. The PXE-enabled NIC of the client sends out a broadcast request to DHCP server, which returns with the IP address of the client along with the address of the TFTP server, and the location of boot files on the TFTP server. The following steps describe how it works:
- Target Machine (either bare metal or with boot sector removed) is booted.
- The Network Interface Card (NIC) of the machine triggers a DHCP request.
- DHCP server intercepts the request and responds with standard information (IP, subnet mask, gateway, DNS etc.). In addition, it provides information about the location of a TFTP server and boot image (pxelinux.0).
- When the client receives this information, it contacts the TFTP server for obtaining the boot image.
- TFTP server sends the boot image (pxelinux.0), and the client executes it.
- By default, the boot image searches the pxelinux.cfg directory on TFTP server for boot configuration files on the TFTP server using the following approach:First, it searches for the boot configuration file that is named according to the MAC address represented in lower case hexadecimal digits with dash separators. For example, for the MAC Address "88:99:AA:BB:CC:DD", it searches for the file 01-88-99-aa-bb-cc-dd.Then, it searches for the configuration file using the IP address (of the machine that is being booted) in upper case hexadecimal digits. For example, for the IP Address "192.0.2.91", it searches for the file "C000025B".If that file is not found, it removes one hexadecimal digit from the end and tries again. However, if the search is still not successful, it finally looks for a file named "default" (in lower case).For example, if the boot file name is /tftpboot/pxelinux.0, the Ethernet MAC address is 88:99:AA:BB:CC:DD, and the IP address 192.0.2.91, the boot image looks for file names in the following order:
/tftpboot/pxelinux.cfg/01-88-99-aa-bb-cc-dd /tftpboot/pxelinux.cfg/C000025B /tftpboot/pxelinux.cfg/C000025 /tftpboot/pxelinux.cfg/C00002 /tftpboot/pxelinux.cfg/C0000 /tftpboot/pxelinux.cfg/C000 /tftpboot/pxelinux.cfg/C00 /tftpboot/pxelinux.cfg/C0 /tftpboot/pxelinux.cfg/C
- The client downloads all the files it needs (kernel and root file system), and then loads them.
- Target Machine reboots.The Provisioning application uses Redhat's Kickstart method to automate the installation of Redhat Linux on target machines. Using kickstart, the system administrator can create a single file containing answers to all the questions that will usually be asked during a typical Red Hat Linux installation.The host specific boot configuration file contains the location of the kickstart file. This kickstart file would have been created earlier by the stage directive of the OS image based on the input from user.
Kickstart sequence of events
http://www.linux-mag.com/id/6747/
A network-based kickstart can be initiated by an PXE Boot capable network card. The PXE Boot first requests an IP address from a DHCP server. It also obtains the location of a PXE Boot file from the DHCP server. PXELINUX is a bootloader for Linux using the PXE network booting protocol. The PXE Boot file is loaded from the TFTP server along with the contents of a file which defines the location and name of the installation kernel and initrd.img file as well as some parameters for the boot kernel and a menu for the Anaconda installer. This configuration file for Anaconda also contains the location of the kickstart configuration file to be used during the installation.
The PXE Boot file then loads the boot kernel and initrd image still using TFTP. After booting, Anaconda is started and Anaconda loads the menu and displays a window with a timer with several menu options. The Menu and time-out can be skipped if you do not need to make any choices here.
After choosing the desired kickstart installation, Anaconda locates the kickstart configuration file from the HTTP server and reads it. The kickstart configuration file has a default name of ks.cfg, but can be named anything. We use several for our different configurations, so provide unique names for each. If all of the data required to perform a complete installation is included in the kickstart configuration file, the installation completes without further intervention from the administrator. The RPM files used during the installation are downloaded from the HTTP server as they are needed.
The kickstart configuration file can also contain
bash script commands that can be run both before and after the rest of the installation. We make extensive use of the post-installation bash scripts to perform installations of locally required RPM packages and tarballs as well as to make configuration changes before the first reboot.Cloud
Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction (NIST)
Why Private Cloud?
Higher security and privacy
More Control
Higher cost efficiency
Higher energy efficiency
Improved reliability
Immutable infrastructure
No lock-in
Why Private Cloud?
Higher security and privacy
More Control
Higher cost efficiency
Higher energy efficiency
Improved reliability
Immutable infrastructure
No lock-in
Monday, April 10, 2017
Friday, April 7, 2017
Design Patterns
https://github.com/faif/python-patterns
Creational , Ploymorphism
Factory : Uncertain in types of objects, Decisions to be made at runtime regarding what classes to use
Singleton: Only one instance, Global variable in an object-oriented way, An information case, shared by multiple objects
Builder: Director, Abstract Builder : interfaces, Concrete Builder: implements the interfaces, Product : object being built, divide and conquer strategy
Prototype: (Creating many identical objects individually is expensive. Cloning is an alternative) Create a prototypical instance first. Simply clone it whenever you need replica.
Structural, Inheritance
Decorator: New features to an existing object, Dynamic changes, Not using subclassing.
Proxy: Postpone object creation unless absolutely necessary. Find a placeholder.
Adapter: Incompatible interfaces
Composite: Recursive tree structure. Component(abstract class), Child (concrete class), Composite(concrete class)
Bridge: Two unrelated, parallel, or orthogonal abstractions. One is implementation specific. The other is implementation independent.
Behavioral, Methods and their signatures
Observer: establishes a one-to-many relationship between a subject and multiple observers. Subjects to be monitored. Observers to be notified.
Visitor: allows adding new features to an existing class hierarchy without changing it.
Iterator: allows a client to have sequential access to the elements of an aggregate object without exposing its underlying structure.
Strategy: offers a family of interchangeable algorithms to a client. The problem we often see is that there is a need for dynamically changing the behavior of an object.
Chain of responsibility: opens up various possibilities of processing for a given request. Decouples the request and its processing.
Coupling and Cohesion
Coupling: The degree to which your software elements are connected
Cohesion: The degree of independence
The goal of design patterns: More cohesion and less coupling
Creational , Ploymorphism
Factory : Uncertain in types of objects, Decisions to be made at runtime regarding what classes to use
Singleton: Only one instance, Global variable in an object-oriented way, An information case, shared by multiple objects
Builder: Director, Abstract Builder : interfaces, Concrete Builder: implements the interfaces, Product : object being built, divide and conquer strategy
Prototype: (Creating many identical objects individually is expensive. Cloning is an alternative) Create a prototypical instance first. Simply clone it whenever you need replica.
Structural, Inheritance
Decorator: New features to an existing object, Dynamic changes, Not using subclassing.
Proxy: Postpone object creation unless absolutely necessary. Find a placeholder.
Adapter: Incompatible interfaces
Composite: Recursive tree structure. Component(abstract class), Child (concrete class), Composite(concrete class)
Bridge: Two unrelated, parallel, or orthogonal abstractions. One is implementation specific. The other is implementation independent.
Behavioral, Methods and their signatures
Observer: establishes a one-to-many relationship between a subject and multiple observers. Subjects to be monitored. Observers to be notified.
Visitor: allows adding new features to an existing class hierarchy without changing it.
Iterator: allows a client to have sequential access to the elements of an aggregate object without exposing its underlying structure.
Strategy: offers a family of interchangeable algorithms to a client. The problem we often see is that there is a need for dynamically changing the behavior of an object.
Chain of responsibility: opens up various possibilities of processing for a given request. Decouples the request and its processing.
Coupling and Cohesion
Coupling: The degree to which your software elements are connected
Cohesion: The degree of independence
The goal of design patterns: More cohesion and less coupling
Monday, April 3, 2017
Security
https://www.ietf.org/rfc/rfc2828.txt
5 services
1 Confidentiality : Protects data against unauthorized disclosure. Use encryption.
2 Integrity: Ensures data has not been changed, destroyed, or lost in an unauthorized or accidental manner. Use a hash function.
3 Accountability: Accountability directly supports non-reputation and provides protection against false denial of involvement in a communication. Use a digital signature.
4 Authentication: Confirm identity of a system entity. Use a Message Authentication Code (MAC)
5 Availability: Services are accessible and available to authorized users or processes in an information infrastructure. Difficult to protect. ex. Denial of Service attack.
Of the five services, four can be achieved by using encryption. That is why cryptography and network security go hand in hand.
5 services
1 Confidentiality : Protects data against unauthorized disclosure. Use encryption.
2 Integrity: Ensures data has not been changed, destroyed, or lost in an unauthorized or accidental manner. Use a hash function.
3 Accountability: Accountability directly supports non-reputation and provides protection against false denial of involvement in a communication. Use a digital signature.
4 Authentication: Confirm identity of a system entity. Use a Message Authentication Code (MAC)
5 Availability: Services are accessible and available to authorized users or processes in an information infrastructure. Difficult to protect. ex. Denial of Service attack.
Of the five services, four can be achieved by using encryption. That is why cryptography and network security go hand in hand.
Thursday, March 9, 2017
Tensorflow intro
https://github.com/hunkim/DeepLearningZeroToAll
https://www.youtube.com/watch?v=g-EvyKpZjmQ
http://web.stanford.edu/class/cs20si/syllabus.html
https://www.youtube.com/watch?v=PicxU81owCs
https://www.youtube.com/watch?v=IgSuFYamZas&list=PLkkuNyzb8LmxFutYuPA7B4oiMn6cjD6Rs
https://www.youtube.com/watch?v=g-EvyKpZjmQ
http://web.stanford.edu/class/cs20si/syllabus.html
Tensorflow for Deep Learning Research - Lecture 2
https://www.youtube.com/watch?v=9kC836XhICUhttps://www.youtube.com/watch?v=PicxU81owCs
MIT 6.S191 Lecture
https://www.youtube.com/watch?v=IgSuFYamZas&list=PLkkuNyzb8LmxFutYuPA7B4oiMn6cjD6Rs
TensorFlow and Deep Learning without a PhD, Part 1 (Google Cloud Next '17)
https://www.youtube.com/watch?v=u4alGiomYP4style transfer
https://github.com/llSourcell/How_to_do_style_transfer_in_tensorflow
https://github.com/llSourcell/How_to_do_style_transfer_in_tensorflow/blob/master/Style_Transfer.ipynb
https://github.com/jcjohnson/neural-style
http://www.makeuseof.com/tag/create-neural-paintings-deepstyle-ubuntu/
https://blog.paperspace.com/art-with-neural-networks/
http://kawahara.ca/deep-dreams-and-a-neural-algorithm-of-artistic-style-slides-and-explanations/
http://www.chioka.in/tensorflow-implementation-neural-algorithm-of-artistic-style
caffe2go
https://code.facebook.com/posts/196146247499076/delivering-real-time-ai-in-the-palm-of-your-hand/
Tuesday, March 7, 2017
Saturday, February 18, 2017
Friday, February 10, 2017
NLP
https://www.youtube.com/watch?v=RP3tZFcC2e8&list=PL613dYIGMXoZBtZhbyiBqb0QtgK6oJbpm
Neural Machine Translation intro
https://arxiv.org/pdf/1703.01619.pdf
Thursday, February 9, 2017
Adversarial GAN
GAN
https://arxiv.org/abs/1406.2661
http://rll.berkeley.edu/adversarial/
http://www.rubedo.com.br/2017/03/generative-adversarial-networks.html
http://aliensunmin.github.io/project/accv16tutorial/media/generative.pdf
InfoGAN
https://arxiv.org/abs/1606.03657
https://github.com/openai/InfoGAN
NIPS 2016
https://nips.cc/Conferences/2016/Videos
Generative Adversarial Networks
How to train a GAN
https://www.youtube.com/watch?v=X1mUN6dD8uE
Saturday, January 14, 2017
tensorflow install + GPU + Ubuntu 16.04
r0.12
sudo pip install tensorflow-gpu
sudo pip install tensorflow-gpu
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64"
export CUDA_HOME=/usr/local/cuda $ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
>>>
Note:Driver mismatch http://stackoverflow.com/questions/42288290/cuda-kernel-doesnt-match-during-installation-of-tensorflow
http://www.developersite.org/906-88688-tensorflow
opencv mask
http://stackoverflow.com/questions/7479265/apply-mask-to-image-in-opencv
inputimg.copyTo(outputimg, mask)
mask non-zero pixel will be copied from input to output
inputimg.copyTo(outputimg, mask)
mask non-zero pixel will be copied from input to output
Sunday, January 1, 2017
segmentation
https://arxiv.org/pdf/1411.4038.pdf
Fully Convolutional Networks for Semantic Segmentation
https://arxiv.org/abs/1611.09326v1
TF Image Segmentation: Image Segmentation framework
https://github.com/warmspringwinds/tf-image-segmentation
https://arxiv.org/abs/1611.09326v1
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
TF Image Segmentation: Image Segmentation framework
https://github.com/warmspringwinds/tf-image-segmentation
Human Pose Estimation
https://arxiv.org/abs/1611.00468
CRF-CNN: Modeling Structured Information in Human Pose Estimation
Subscribe to:
Posts (Atom)