Thursday, 13 April 2017

Gauva-Retryer

Retrier:  The guava-retrying module provides a [general purpose method] for retrying java arbitary code with specific stop,retry and exception handling capabilities that are enhanced by Gauva's predicate matching.

1) API calls,Network calls are failing are normal part of our programs. We handle them in Application specific ways.
2) Many times,we need to [retry] the operations before logging it as failure.
3) Data Fetcher ----------> Remote Server
   For demonstration, we have a Remote Server from where we need to fetch the data.
   It could be [streaming] the data or a [REST API] call or any other [RPC] call.

   static class UrlFetcher implements Callable<Boolean>{
     private String url;
     private String opMode;

     UrlFetcher(String url,String opMode){
      this.url = url;
      this.opMode = opMode;
     }

     @Override
     public Boolean call() throws Exception{
      if("fail".equals(opMode)){
       throw new TimeoutException("Connection timed out");
      }

      return true;
     }
   }

   Retryer<Boolean> retrier = RetryerBuilder.<Boolean>newBuilder()
                              .RetryIfExceptionOfType(TimeoutException.class)
                              .retryIfRuntimeException()
                              .withStopStrategy(StopStrategies.stopAfterAttempt(3))
                              .build();

  try{
    retrier.call(new UrlFetcher("http://www.google.com","normal"));
    retrier.call(new UrlFetcher("http://www.google.com","fail"));
  } catch(RetryException e){
        e.printStackTrace();
  } catch(ExecutionException e){
        e.printStackTrace();
  }

 4) a) We need to create retryer using builder.
    b) We can specify the conditons on which to retry.
    c) Here we have specified RuntimeException and TimeoutException, if there is any exception,it won't retry.
    d) We also need to specify strategy to stop. Here we have specified,retry 3 times and then abort.

5) Wait Strategy:
------------------
1) Here we retried immediately after failure, what if we want to change the behaviour.
.withWaitStrategy(WaitStrategies.exponentailWait(1000,5,TimeUnit.SECONDS))

Tuesday, 11 April 2017

Maven Notes

  Maven:    No need to write build script and manage dependencies
----------
Project management and comprehension tool based on concept of [POM] --> Project Object Model.

Maven -Manages--> [Build,Dependecies,SCMs] Releases,distribution, Reporting and Documentation.

a) Maven provides developers a [complete build life cycle framework].
b) Maven uses [**standard directory layout and ** default build cycle].
c) Maven project structure and contents are mentioned in pom.xml[project object model].
d) When Maven project created, maven creates a standard project structure, deveopers are required only to place files in those folders.

e) ${basedir}/src/main/java/ -> Source code.
   ${basedir}/src/main/resources --> resources.
   ${basedir}/src/test --> Test code.
   ${basedir}/src/test/resources --> Test resources.
   ${basedir}/src/target --> distibutable jar.
   ${basedir}/src/target/classes --> compiled class files.

7) In order to build the project, maven provides developers options to mention [life-cycle goals] and [project dependencies](in pom.xml).

8) Export M2_HOME=export M2_HOME=/usr/local/apache-maven/apache-maven-3.3.3 and add bin folder to PATH to run maven build from anywhere in sys.

9)POM.xml -> It's XML file
  a) Always resides in [base directory] of project.
  b) While executing maven task/goal, maven looks for pom.xml in current directory.
  c) POM might contains below information:
     a) project dependencies b) plugins(compiler plugin etc) c) goals(custom goals) d) build profiles (dev,prod,test) e) project version(under project element witch is root element) f) developers d) mailing list

     We can modify, project folder structure

**d) Before creating POM, we need to come up with project group(groupid),project name(artifact) and version for [unique] identification of project in repository.

  e) It's allowed only one POM file per project.
  f) All POMs inherit from a parent POM known as super POM.
  g) Maven provides different archtypes to ssupport diff project structures.

Build Life cycle:
-------------------

Build Life Cycle Contains Phases --> Phases contains Goals

1) Typical build cycle contains following phases:
   a) prepare-resources (Resource copying can be customized in this phase.) b) compile (just compile source) c) package (build jar/war file) d) install (upload artifacts to local/remote repository)

2) Every phase has pre and post phases to register goals.

c) Maven has 3 standard life cycles: a) clean b) default or build d) site (Documentation)

d) Goal is a specific task which contributes to the build.
e) Order of invocation matters

   clean phase -> dependency goal -> package phase
   mvn clean dependency:copy-dependencies package
         |         |                         |
       Phase ->   Goal                 ->   Phase

f) Default/Build life cycle : Has 23 phases

Build Profiles:
----------------
a) Build profiles can be used to distinguish build definations for diff env like prod,dev and UAT
b) Build profiles are three types:
   Per Project -> Defined in project pom.xml
   Per User -> Defined in Users/username/.m2/settings.xml
   Global -> %M2_HOME%/conf/settings.xml

   mvn phase -Pprod
c) Profile can be activated in pom.xml using activeProfiles tag.

Repository:
----------------
a) Repository is a place, all project jars,library jars,plugins resides.
b) 3 types: Local,Central and Remote
Local: i)   Gets created in %USER_HOME% folder.
       ii)  Maven keeps all project dependencies in local repository when mvn build runs.
       iii) Path can be overridden by mentioning path in settings.xml in %M2_HOME%/conf folder.

Central: Provoded by maven community; http://repo1.maven.org/maven/

Remote: Developers own custom library

Order of reference for dependencies: Local -> Central -> Remote

Plugins:
------------

1) Maven is actually plugin execution framework where every task is done by plugins
2) Ex: Creating jar file,create war file,compile source code,unit testing code,create documentation and project reports
3) Plugins usually provides multiple goals can be executed like this: [mvn plugin-name:goal-name]
   For ex: Java project can be compiled by using mvn-compiler-plugin's mvn compiler:compile
4) PLugins are two types: Build plugins (under /build) and reporting plugins(under /reporting)
5) in POM.xml <plugins><plugin><executions><execution><goals><goal></goal><goals><configuration><tasks></tasks></configuration><executions><execution></plugin></plugins>

Creating Projects:
-------------------
1) Maven uses archtype plugin to create projects.

External Dependencies:
-----------------------

We can add lib folder with jars to maven to refer when those jars are not found in all 3 repos.
<dependency><groupid></groupid><artifactid></artifactid><version></version><scope>system</scope><systempath></systempath></dependency>

-U to force maven to download latest dependencies

****** if version contains snapshot, it means maven fetch copy from repo for every build. unlike normal version case
Build automation: Kick off build of dependent projects, once current build is done successfully.
This can be acieved by modifying pom.xml or else using continuous integration(CI) hudson
Whenever SVN check in happens, it will trigger build and then triggers dependent builds.

Transitive Dependencies Discovery:
----------------------------------------------
It is pretty often a case, when a library say A depends upon other library say B. In case another project C want to use A then that project requires to use library B too.

Maven helps to avoid such requirement to discover all the libraries required. Maven does so by reading project files(pom.xml) of dependencies, figure out their dependencies and so on.

We only need to define [direct dependency] in each project pom. Maven handles the rest automatically.

Common dependencies can be placed at single place using concept of parent pom.Dependencies

Scope Description
compile This scope indicates that dependency is available in classpath of project. It is default scope.
provided This scope indicates that dependency is to be provided by JDK or web-Server/Container at runtime
runtime This scope indicates that dependency is not required for compilation, but is required during execution.
test This scope indicates that the dependency is only available for the test compilation and execution phases.
system This scope indicates that you have to provide the system path.
import This scope is only used when dependency is of type pom. This scopes indicates that the specified POM should be replaced with the dependencies in that POM's <dependencyManagement> section.

mvn dependency:tree

Monday, 13 February 2017

Running jobs in background using nohup

We can use nohup to start jobs in background.

 Usage: nohup script.name > log file name 2>&1(send error log also to normal log) &

in the end, & is mandatory.

Thursday, 5 January 2017

File Permissions in UNIX

rwx(owner user)-rwx(group)-rwx(other)

0 - no permission
1 -> x only
2 -> w only
3 -> wx only
4 -> r only
5 -> rx only
6 -> rw only
7 -> rwx - all


chmod -R 777 to change file permissions.




Wednesday, 4 January 2017

Beauty Of Grep

Usage:grep keyword filename

1) How to ignore some words while doing search using grep in UNIX:

Example:
grep Exception logfile.txt | grep -v Error

2) How to count occurrence of a word in a file using grep command:

grep -c Exception logfile.txt



3) Printing lines after  or before matching a word :

grep -C 6 Exception logfile.txt



4) How to search pattern using egrep and regular expression:

We can use '|' to search for either Error or Exception in single comand

egrep 'Error|Exception' filename.txt



5) -i can be used for case insensitive search

6) zgrep can be used to search patterns in gz files.

7) How to search a whole word in file using grep

grep -w word filename









Tuesday, 3 January 2017

How To Find Out List Of All Open Ports in Linux

Network Statistics(Netstat) can be used to identify all incoming and outgoing connections in a system.

List out all connections: netstat -a




If we want to display only tcp connections, then 't' option can be used to along with option a.
Similarly 'u' option to display UDP connection details.

When viewing the open/listening port and connections,it's often useful to know the process name/pid which has opened that port or connection.

Note: When using p option, netstat must be run with root privileges.

n -> No reverse dns lookup to speed up the output.
l --> Display only listening(established) connections.
t --> Interested only in tcp connections.
p --> Display who opened those connections.
e --> Display user id




Check if a service is running: use grep with pipe

 

Print netstat output continuously (option c)

netstat -cnt | grep ESTABLISHED













FIND Any File or Directory in LINUX

Find command can be used to locate files in LINUX.

Default Usage:

find directory  -name name