<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V2.0//EN" "http://forrest.apache.org/dtd/document-v20.dtd">

<document>
  <header>
    <title>Example </title>
  </header>
  <body>
   <p>The following example, extracted from the HCatalog documentation, shows how people
    might use HCatalog along with various other Hadoop tools to move data from the grid
    into a database and ultimately analyze it.</p>

   <p><strong>Without Templeton</strong> there are three main steps to completing
     the task.</p>

   <p>First, Joe in data acquisition uses distcp to get data
      onto the grid.</p>

<source>
hadoop distcp file:///file.dat hdfs://data/rawevents/20100819/data

hcat "alter table rawevents add partition 20100819 hdfs://data/rawevents/20100819/data"
</source>

<p>Second, Sally in data processing uses Pig to cleanse and prepare the
    data.  Oozie will be notified by HCatalog that data is available and can then
    start the Pig job</p>

<source>
A = load 'rawevents' using HCatLoader;
B = filter A by date = '20100819' and by bot_finder(zeta) = 0;
…
store Z into 'processedevents' using HCatStorer("date=20100819");
</source>

<p>Third, Robert in client management uses Hive to analyze his
   clients' results.</p>

<source>
insert overwrite table 20100819events
select advertiser_id, count(clicks)
from processedevents
where date = ‘20100819’
group by advertiser_id;
</source>

<p><strong>With Templeton</strong> all these steps can be easily performed programatcally
 upon receipt of the initial data.  Sally and Robert can still maintain their own scripts
 and simply push them into HDFS to be accessed when required by Templeton. </p>

<source>
??Still need to add web hdfs push!

>POST /v1/templeton/ddl.json?exec="alter table rawevents add partition 20100819 hdfs://data/rawevents/20100819/data"
>{"result":"ok"}
>
>POST /v1/templeton/pig.json?src="hdfs://scripts/cleanse.pig"
>{"result": "ok", "jobid": "123"}
>
>...
>GET /v1/templeton/queue/123.json
>{"result": "ok", "status" "completed"}
>
>POST /v1/templeton/hive.json?src="hdfs://scripts/analyze.hive"
>{"result": "ok", "jobid": "456"}
>
</source>

  </body>
</document>
