org.apache.hadoop.mapred

Class SequenceFileAsBinaryOutputFormat

java.lang.Object

org.apache.hadoop.mapred.FileOutputFormat<K,V>

org.apache.hadoop.mapred.SequenceFileOutputFormat<BytesWritable,BytesWritable>

org.apache.hadoop.mapred.SequenceFileAsBinaryOutputFormat

All Implemented Interfaces:

OutputFormat<BytesWritable,BytesWritable>

@InterfaceAudience.Public
 @InterfaceStability.Stable
public class SequenceFileAsBinaryOutputFormat
extends SequenceFileOutputFormat<BytesWritable,BytesWritable>

An OutputFormat that writes keys, values to SequenceFiles in binary(raw) format

Constructor Summary

Constructors

Constructor and Description
SequenceFileAsBinaryOutputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(FileSystem ignored, JobConf job) Check for validity of the output-specification for the job.
RecordWriter<BytesWritable,BytesWritable>	getRecordWriter(FileSystem ignored, JobConf job, String name, Progressable progress) Get the RecordWriter for the given job.
static Class<? extends WritableComparable>	getSequenceFileOutputKeyClass(JobConf conf) Get the key class for the SequenceFile
static Class<? extends Writable>	getSequenceFileOutputValueClass(JobConf conf) Get the value class for the SequenceFile
static void	setSequenceFileOutputKeyClass(JobConf conf, Class<?> theClass) Set the key class for the SequenceFile
static void	setSequenceFileOutputValueClass(JobConf conf, Class<?> theClass) Set the value class for the SequenceFile

Methods inherited from class org.apache.hadoop.mapred.SequenceFileOutputFormat

getOutputCompressionType, getReaders, setOutputCompressionType

Methods inherited from class org.apache.hadoop.mapred.FileOutputFormat

getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SequenceFileAsBinaryOutputFormat

public SequenceFileAsBinaryOutputFormat()

Method Detail

setSequenceFileOutputKeyClass

public static void setSequenceFileOutputKeyClass(JobConf conf,
                                                 Class<?> theClass)

Set the key class for the SequenceFile

This allows the user to specify the key class to be different from the actual class (BytesWritable) used for writing

Parameters:

conf - the JobConf to modify

theClass - the SequenceFile output key class.

setSequenceFileOutputValueClass

public static void setSequenceFileOutputValueClass(JobConf conf,
                                                   Class<?> theClass)

Set the value class for the SequenceFile

This allows the user to specify the value class to be different from the actual class (BytesWritable) used for writing

Parameters:

conf - the JobConf to modify

theClass - the SequenceFile output key class.

getSequenceFileOutputKeyClass

public static Class<? extends WritableComparable> getSequenceFileOutputKeyClass(JobConf conf)

Get the key class for the SequenceFile

Returns:

the key class of the SequenceFile

getSequenceFileOutputValueClass

public static Class<? extends Writable> getSequenceFileOutputValueClass(JobConf conf)

Get the value class for the SequenceFile

Returns:

the value class of the SequenceFile

getRecordWriter

public RecordWriter<BytesWritable,BytesWritable> getRecordWriter(FileSystem ignored,
                                                                 JobConf job,
                                                                 String name,
                                                                 Progressable progress)
                                                          throws IOException

Description copied from interface: OutputFormat

Get the RecordWriter for the given job.

Specified by:

getRecordWriter in interface OutputFormat<BytesWritable,BytesWritable>

Overrides:

getRecordWriter in class SequenceFileOutputFormat<BytesWritable,BytesWritable>

job - configuration for the job whose output is being written.

name - the unique name for this part of the output.

progress - mechanism for reporting progress while writing to file.

Returns:

a RecordWriter to write the output for the job.

Throws:

IOException

checkOutputSpecs

public void checkOutputSpecs(FileSystem ignored,
                             JobConf job)
                      throws IOException

Description copied from interface: OutputFormat

Check for validity of the output-specification for the job.

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

Implementations which write to filesystems which support delegation tokens usually collect the tokens for the destination path(s) and attach them to the job configuration.

Specified by:

checkOutputSpecs in interface OutputFormat<BytesWritable,BytesWritable>

Overrides:

checkOutputSpecs in class FileOutputFormat<BytesWritable,BytesWritable>

job - job configuration.

Throws:

IOException - when output should not be attempted