org.apache.ignite.internal.processors.hadoop.impl.igfs

Class HadoopIgfsOutProc

java.lang.Object

org.apache.ignite.internal.processors.hadoop.impl.igfs.HadoopIgfsOutProc

All Implemented Interfaces:

HadoopIgfs, HadoopIgfsEx, HadoopIgfsIpcIoListener

public class HadoopIgfsOutProc
extends Object
implements HadoopIgfsEx, HadoopIgfsIpcIoListener

Communication with external process (TCP or shmem).

Constructor Summary

Constructors

Constructor and Description
HadoopIgfsOutProc(int port, String igfs, org.apache.commons.logging.Log log, String user) Constructor for shmem endpoint.
HadoopIgfsOutProc(String host, int port, String igfs, org.apache.commons.logging.Log log, String user) Constructor for TCP endpoint.

Method Summary

Modifier and Type	Method and Description
void	addEventListener(HadoopIgfsStreamDelegate desc, HadoopIgfsStreamEventListener lsnr) Adds event listener that will be invoked when connection with server is lost or remote error has occurred.
Collection<IgfsBlockLocation>	affinity(IgfsPath path, long start, long len) Command to get affinity for given path, offset and length.
HadoopIgfsStreamDelegate	append(IgfsPath path, boolean create, Map<String,String> props) Open file for output appending data to the end of a file.
void	close(boolean force) Close connection.
void	closeStream(HadoopIgfsStreamDelegate desc) Close server stream.
IgfsPathSummary	contentSummary(IgfsPath path) Gets path summary.
HadoopIgfsStreamDelegate	create(IgfsPath path, boolean overwrite, boolean colocate, int replication, long blockSize, Map<String,String> props) Command to create file and open it for output.
Boolean	delete(IgfsPath path, boolean recursive) Command to delete given path.
void	flush(HadoopIgfsStreamDelegate delegate) Flush output stream.
IgfsStatus	fsStatus() Performs status request.
IgfsHandshakeResponse	handshake(String logDir) Perform handshake.
IgfsFile	info(IgfsPath path) Command to retrieve file info for some IGFS path.
Collection<IgfsFile>	listFiles(IgfsPath path) Command to get list of files in directory.
Collection<IgfsPath>	listPaths(IgfsPath path) Command to get directory listing.
Boolean	mkdirs(IgfsPath path, Map<String,String> props) Command to create directories.
IgfsModeResolver	modeResolver()
void	onClose() Callback invoked when the IO is being closed.
void	onError(long streamId, String errMsg) Callback invoked when remote error occurs.
HadoopIgfsStreamDelegate	open(IgfsPath path) Command to open file for reading.
HadoopIgfsStreamDelegate	open(IgfsPath path, int seqReadsBeforePrefetch) Command to open file for reading.
IgniteInternalFuture<byte[]>	readData(HadoopIgfsStreamDelegate desc, long pos, int len, byte[] outBuf, int outOff, int outLen) Asynchronously reads specified amount of bytes from opened input stream.
void	removeEventListener(HadoopIgfsStreamDelegate desc) Removes event listener that will be invoked when connection with server is lost or remote error has occurred.
Boolean	rename(IgfsPath src, IgfsPath dest) Command to rename given path.
Boolean	setTimes(IgfsPath path, long accessTime, long modificationTime) Sets last access time and last modification time for a file.
IgfsFile	update(IgfsPath path, Map<String,String> props) Command to update file properties.
String	user() The user this Igfs instance works on behalf of.
void	writeData(HadoopIgfsStreamDelegate desc, byte[] data, int off, int len) Writes data to the stream with given streamId.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopIgfsOutProc

public HadoopIgfsOutProc(String host,
                         int port,
                         String igfs,
                         org.apache.commons.logging.Log log,
                         String user)
                  throws IOException

Constructor for TCP endpoint.

Parameters:

host - Host.

port - Port.

igfs - IGFS name.

log - Client logger.

user - User name.

Throws:

IOException - If failed.

HadoopIgfsOutProc

public HadoopIgfsOutProc(int port,
                         String igfs,
                         org.apache.commons.logging.Log log,
                         String user)
                  throws IOException

Constructor for shmem endpoint.

Parameters:

port - Port.

igfs - IGFS name.

log - Client logger.

user - User name.

Throws:

IOException - If failed.

Method Detail

handshake

public IgfsHandshakeResponse handshake(String logDir)
                                throws IgniteCheckedException

Perform handshake.

Specified by:

handshake in interface HadoopIgfs

Parameters:

logDir - Log directory.

Returns:

Future with handshake result.

Throws:

IgniteCheckedException - If failed.

close

public void close(boolean force)

Close connection.

Specified by:

close in interface HadoopIgfs

Parameters:

force - Force flag.

info

public IgfsFile info(IgfsPath path)
              throws IgniteCheckedException

Command to retrieve file info for some IGFS path.

Specified by:

info in interface HadoopIgfs

Parameters:

path - Path to get file info for.

Returns:

Future for info operation.

Throws:

IgniteCheckedException - If failed.

update

public IgfsFile update(IgfsPath path,
                       Map<String,String> props)
                throws IgniteCheckedException

Command to update file properties.

Specified by:

update in interface HadoopIgfs

Parameters:

path - IGFS path to update properties.

props - Properties to update.

Returns:

Future for update operation.

Throws:

IgniteCheckedException - If failed.

setTimes

public Boolean setTimes(IgfsPath path,
                        long accessTime,
                        long modificationTime)
                 throws IgniteCheckedException

Sets last access time and last modification time for a file.

Specified by:

setTimes in interface HadoopIgfs

Parameters:

path - Path to update times.

accessTime - Last access time to set.

modificationTime - Last modification time to set.

Throws:

IgniteCheckedException - If failed.

rename

public Boolean rename(IgfsPath src,
                      IgfsPath dest)
               throws IgniteCheckedException

Command to rename given path.

Specified by:

rename in interface HadoopIgfs

Parameters:

src - Source path.

dest - Destination path.

Returns:

Future for rename operation.

Throws:

IgniteCheckedException - If failed.

delete

public Boolean delete(IgfsPath path,
                      boolean recursive)
               throws IgniteCheckedException

Command to delete given path.

Specified by:

delete in interface HadoopIgfs

Parameters:

path - Path to delete.

recursive - True if deletion is recursive.

Returns:

Future for delete operation.

Throws:

IgniteCheckedException - If failed.

affinity

public Collection<IgfsBlockLocation> affinity(IgfsPath path,
                                              long start,
                                              long len)
                                       throws IgniteCheckedException

Command to get affinity for given path, offset and length.

Specified by:

affinity in interface HadoopIgfs

Parameters:

path - Path to get affinity for.

start - Start position (offset).

len - Data length.

Returns:

Future for affinity command.

Throws:

IgniteCheckedException - If failed.

contentSummary

public IgfsPathSummary contentSummary(IgfsPath path)
                               throws IgniteCheckedException

Gets path summary.

Specified by:

contentSummary in interface HadoopIgfs

Parameters:

path - Path to get summary for.

Returns:

Future that will be completed when summary is received.

Throws:

IgniteCheckedException - If failed.

mkdirs

public Boolean mkdirs(IgfsPath path,
                      Map<String,String> props)
               throws IgniteCheckedException

Command to create directories.

Specified by:

mkdirs in interface HadoopIgfs

Parameters:

path - Path to create.

Returns:

Future for mkdirs operation.

Throws:

IgniteCheckedException - If failed.

listFiles

public Collection<IgfsFile> listFiles(IgfsPath path)
                               throws IgniteCheckedException

Command to get list of files in directory.

Specified by:

listFiles in interface HadoopIgfs

Parameters:

path - Path to list.

Returns:

Future for listFiles operation.

Throws:

IgniteCheckedException - If failed.

listPaths

public Collection<IgfsPath> listPaths(IgfsPath path)
                               throws IgniteCheckedException

Command to get directory listing.

Specified by:

listPaths in interface HadoopIgfs

Parameters:

path - Path to list.

Returns:

Future for listPaths operation.

Throws:

IgniteCheckedException - If failed.

fsStatus

public IgfsStatus fsStatus()
                    throws IgniteCheckedException

Performs status request.

Specified by:

fsStatus in interface HadoopIgfs

Returns:

Status response.

Throws:

IgniteCheckedException - If failed.

open

public HadoopIgfsStreamDelegate open(IgfsPath path)
                              throws IgniteCheckedException

Command to open file for reading.

Specified by:

open in interface HadoopIgfs

Parameters:

path - File path to open.

Returns:

Future for open operation.

Throws:

IgniteCheckedException - If failed.

open

public HadoopIgfsStreamDelegate open(IgfsPath path,
                                     int seqReadsBeforePrefetch)
                              throws IgniteCheckedException

Command to open file for reading.

Specified by:

open in interface HadoopIgfs

Parameters:

path - File path to open.

Returns:

Future for open operation.

Throws:

IgniteCheckedException - If failed.

create

public HadoopIgfsStreamDelegate create(IgfsPath path,
                                       boolean overwrite,
                                       boolean colocate,
                                       int replication,
                                       long blockSize,
                                       @Nullable
                                       Map<String,String> props)
                                throws IgniteCheckedException

Command to create file and open it for output.

Specified by:

create in interface HadoopIgfs

Parameters:

path - Path to file.

overwrite - If true then old file contents will be lost.

colocate - If true and called on data node, file will be written on that node.

replication - Replication factor.

props - File properties for creation.

Returns:

Stream descriptor.

Throws:

IgniteCheckedException - If failed.

append

public HadoopIgfsStreamDelegate append(IgfsPath path,
                                       boolean create,
                                       @Nullable
                                       Map<String,String> props)
                                throws IgniteCheckedException

Open file for output appending data to the end of a file.

Specified by:

append in interface HadoopIgfs

Parameters:

path - Path to file.

create - If true, file will be created if does not exist.

props - File properties.

Returns:

Stream descriptor.

Throws:

IgniteCheckedException - If failed.

readData

public IgniteInternalFuture<byte[]> readData(HadoopIgfsStreamDelegate desc,
                                             long pos,
                                             int len,
                                             @Nullable
                                             byte[] outBuf,
                                             int outOff,
                                             int outLen)

Asynchronously reads specified amount of bytes from opened input stream.

Specified by:

readData in interface HadoopIgfsEx

Parameters:

desc - Stream delegate.

pos - Position to read from.

len - Data length to read.

outBuf - Optional output buffer. If buffer length is less then len, all remaining bytes will be read into new allocated buffer of length {len - outBuf.length} and this buffer will be the result of read future.

outOff - Output offset.

outLen - Output length.

Returns:

Read data.

writeData

public void writeData(HadoopIgfsStreamDelegate desc,
                      byte[] data,
                      int off,
                      int len)
               throws IOException

Writes data to the stream with given streamId. This method does not return any future since no response to write request is sent.

Specified by:

writeData in interface HadoopIgfsEx

Parameters:

desc - Stream delegate.

data - Data to write.

off - Offset.

len - Length.

Throws:

IOException - If failed.

flush

public void flush(HadoopIgfsStreamDelegate delegate)
           throws IOException

Flush output stream.

Specified by:

flush in interface HadoopIgfsEx

Parameters:

delegate - Stream delegate.

Throws:

IOException - If failed.

closeStream

public void closeStream(HadoopIgfsStreamDelegate desc)
                 throws IOException

Close server stream.

Specified by:

closeStream in interface HadoopIgfsEx

Parameters:

desc - Stream delegate.

Throws:

IOException - If failed.

addEventListener

public void addEventListener(HadoopIgfsStreamDelegate desc,
                             HadoopIgfsStreamEventListener lsnr)

Adds event listener that will be invoked when connection with server is lost or remote error has occurred. If connection is closed already, callback will be invoked synchronously inside this method.

Specified by:

addEventListener in interface HadoopIgfsEx

Parameters:

desc - Stream delegate.

lsnr - Event listener.

removeEventListener

public void removeEventListener(HadoopIgfsStreamDelegate desc)

Removes event listener that will be invoked when connection with server is lost or remote error has occurred.

Specified by:

removeEventListener in interface HadoopIgfsEx

Parameters:

desc - Stream delegate.

onClose

public void onClose()

Callback invoked when the IO is being closed.

Specified by:

onClose in interface HadoopIgfsIpcIoListener

onError

public void onError(long streamId,
                    String errMsg)

Callback invoked when remote error occurs.

Specified by:

onError in interface HadoopIgfsIpcIoListener

Parameters:

streamId - Stream ID.

errMsg - Error message.

user

public String user()

The user this Igfs instance works on behalf of.

Specified by:

user in interface HadoopIgfsEx

Returns:

the user name.

modeResolver

public IgfsModeResolver modeResolver()
                              throws IgniteCheckedException

Specified by:

modeResolver in interface HadoopIgfsEx

Returns:

Mode resolver.

Throws:

IgniteCheckedException - On error.