org.apache.ignite.internal.processors.hadoop.impl.fs

Class HadoopFileSystemCacheUtils

java.lang.Object

org.apache.ignite.internal.processors.hadoop.impl.fs.HadoopFileSystemCacheUtils

public class HadoopFileSystemCacheUtils
extends Object

File system cache utility methods used by Map-Reduce tasks and jobs.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HadoopFileSystemCacheUtils.FsCacheKey Note that configuration is not a part of the key.

Constructor Summary

Constructors

Constructor and Description
HadoopFileSystemCacheUtils()

Method Summary

Modifier and Type	Method and Description
static HadoopLazyConcurrentMap<HadoopFileSystemCacheUtils.FsCacheKey,FileSystem>	createHadoopLazyConcurrentMap() A common static factory method.
static FileSystem	fileSystemForMrUserWithCaching(URI uri, Configuration cfg, HadoopLazyConcurrentMap<HadoopFileSystemCacheUtils.FsCacheKey,FileSystem> map) Common method to get the V1 file system in MapRed engine.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopFileSystemCacheUtils

public HadoopFileSystemCacheUtils()

Method Detail

createHadoopLazyConcurrentMap

public static HadoopLazyConcurrentMap<HadoopFileSystemCacheUtils.FsCacheKey,FileSystem> createHadoopLazyConcurrentMap()

A common static factory method. Creates new HadoopLazyConcurrentMap.

Returns:

a new HadoopLazyConcurrentMap.

fileSystemForMrUserWithCaching

public static FileSystem fileSystemForMrUserWithCaching(@Nullable
                                                        URI uri,
                                                        Configuration cfg,
                                                        HadoopLazyConcurrentMap<HadoopFileSystemCacheUtils.FsCacheKey,FileSystem> map)
                                                 throws IOException

Common method to get the V1 file system in MapRed engine. It gets the filesystem for the user specified in the configuration with MRJobConfig.USER_NAME property. The file systems are created and cached in the given map upon first request.

Parameters:

uri - The file system uri.

cfg - The configuration.

map - The caching map.

Returns:

The file system.

Throws:

IOException - On error.