Projekt „Nový hledač“

File systém - zadání

WWW odkazy - ZFS

Podmínky

Hledám File System, který:

- má ovládání z Ubuntu

- má co největší níže uvedené počty

- je co nejrychlejší, především při reálných velkých níže uvedených počtech

- má kompresi (gzip, ZFS)

Počty

1. Maximální počet adresářů na jednom disku (diskovém poli).

2. Maximální počet souborů na jednom disku (diskovém poli).

3. Maximální počet podadresářů v jednom adresáři.

4. Maximální počet souborů v jednom adresáři.

Preference

ZFS

Další možnosti

btrfs , ext2, ext3, ext4, Lustre, NFTS, XFS (plus další).

WWW odkazy

Obecné

http://www.google.cz/#sclient=psy&hl=cs&q=file+system
http://en.wikipedia.org/wiki/File_system
http://cs.wikipedia.org/wiki/File_system
Software, který realizuje souborový systém, bývá obvykle součástí operačního systému. Většina operačních systémů podporuje několik různých souborových systémů. V Microsoft Windows nalezneme podporu pro souborové systémy FAT a NTFS a ISO 9660 pro ukládání souborů na CD a DVD. V Linuxu nalezneme kromě již zmíněných také ext2, ext3, ext4, ReiserFS, JFS, XFS a mnoho dalších. DOS podporuje systémy FAT, po instalaci CD/DVD driveru také ISO 9660. Solaris podporuje především UFS a ZFS, ale i mnoho dalších.

http://www.google.cz/search?hl=cs&biw=1276&bih=762&q=File+system+Maximum+number+of+files+and+subfolders+within+a+single+folder
http://ph.answers.yahoo.com/question/index?qid=20100606201347AAiAx5H
http://en.wikipedia.org/wiki/List_of_file_systems
http://en.wikipedia.org/wiki/Comparison_of_file_systems
http://technet.microsoft.com/en-us/library/bb457112.aspx
http://en.wikipedia.org/wiki/Journaling_file_system
http://en.wikipedia.org/wiki/Loadable_Kernel_Module
http://en.wikipedia.org/wiki/Copy-on-write
http://en.wikipedia.org/wiki/Cluster_File_Systems
http://www.mcs.anl.gov/research/projects/mpich2/

ZFS

http://www.google.cz/#sclient=psy&hl=cs&q=ZFS
http://en.wikipedia.org/wiki/ZFS
Linux via ZFS-FUSE or partial native support via 3rd party kernel module.
Ubuntu – see kqstor.

http://cs.wikipedia.org/wiki/ZFS
ZFS is not a native cluster, distributed, or parallel file system and cannot provide concurrent access from multiple hosts as ZFS is a local file system. Sun's Lustre distributed filesystem will adapt ZFS as back-end storage for both data and metadata in version 3.0, which is scheduled to be released in 2010.

ZFS has no defragmentation utility. Usage of copy-on-write with often changed files leads to high fragmentation.

Variable block sizes

ZFS uses variable-sized blocks of up to 128 kilobytes. The currently available code allows the administrator to tune the maximum block size used as certain workloads do not perform well with large blocks. Automatic tuning to match workload characteristics is contemplated.

If data compression (LZJB) is enabled, variable block sizes are used. If a block can be compressed to fit into a smaller block size, the smaller size is used on the disk to use less storage and improve IO throughput (though at the cost of increased CPU use for the compression and decompression operations).

http://hub.opensolaris.org/bin/view/Community+Group+zfs/

http://www.root.cz/n/zfs/

http://www.root.cz/vyhledavani/?cx=016098403239631414888%3A7yxfxng7aus&cof=FORID%3A9&ie=UTF-8&q=ZFS&sa=Hledat&siteurl=www.root.cz%2F#762
http://www.root.cz/clanky/co-umi-souborovy-system-zfs/

http://www.root.cz/clanky/zfs-je-v-linuxu-k-dispozici-diky-fuse/

http://www.root.cz/clanky/suborovy-system-zfs-konzistentnost-dat/

http://www.root.cz/clanky/zfs-vytvarame-a-upravujeme-zaklad-pre-nase-data/

http://www.root.cz/clanky/snimky-suboroveho-systemu-zfs/

http://www.root.cz/zpravicky/zfs-dostal-podporu-deduplikace/

http://www.root.cz/zpravicky/nativni-zfs-pro-linux-mozna-uz-za-mesic/

http://www.root.cz/zpravicky/jak-je-na-tom-vykon-nativniho-zfs/

http://www.root.cz/zpravicky/nativni-zfs-pro-linux-ke-stazeni/

http://www.root.cz/zpravicky/vyslo-zfs-fuse-0-7-0/

https://launchpad.net/~bugs-sehe/+archive/zfs-fuse/

http://www.phoronix.com/scan.php?page=news_item&px=ODgyNA

http://kqstor.com
http://kqstor.com/Storage.aspx?page=storage

http://kqstor.com/virtualization.aspx?page=virtualization

http://kqstor.com/product.aspx

http://kqstor.com/downloadlist.aspx?page=download

http://www.kqstor.com/DownloadList.aspx

Pozor, jsou nefunkční odkazy na native ZFS in Ubuntu:
http://kqstor.com
http://kqstor.com/Storage.aspx?page=storage
http://kqstor.com/virtualization.aspx?page=virtualization
http://kqstor.com/product.aspx
http://kqstor.com/downloadlist.aspx?page=download
http://www.kqstor.com/DownloadList.aspx
Jessica Joseph
Business Development
Skype: kqiweb
KQ Infotech
www.kqinfotech.com
unnati@kqinfotech.com
Poslal jsme jí email.

Btrfs

http://www.google.cz/#hl=cs&xhr=t&q=Btrfs

http://en.wikipedia.org/wiki/Btrfs

Linux.

In 2008 the principal developer of the ext3 and ext4 file systems, Theodore Ts'o, stated that ext4 is a stop-gap and that Btrfs is the way forward, having "a number of the same design ideas that reiser3/4 had". While it was merged into the mainline kernel, there are also differing opinions on Btrfs stability. Edward Shishkin, one of the Reiser4 developers now working for Red Hat, has called Btrfs "completely broken" but not responded to clarifications.
http://cs.wikipedia.org/wiki/Btrfs

https://btrfs.wiki.kernel.org/index.php/Main_Page

https://btrfs.wiki.kernel.org/index.php/FAQ

http://oss.oracle.com/projects/btrfs/

http://www.root.cz/clanky/btrfs-zajimavy-ale-nekompletni/

http://thasulinux.wordpress.com/2010/12/14/an-experiment-with-btrfs/

https://lkml.org/lkml/2010/6/18/144

hey, where are other 83% of my disk space???

In the meanwhile I confirm that Btrfs design is completely broken: records stored in the B-tree differ greatly from each other (it is unacceptable!), and the balancing algorithms have been modified in insane manner. All these factors has led to loss of *all* boundaries holding internal fragmentation and to exhaustive waste of disk space (and memory!) in spite of the property "scaling in their ability to address large storage.

It seems that nobody have reviewed Btrfs before its inclusion to the mainline. I have only found a pair of recommendations with a common idea that Btrfs maintainer is "not a crazy man". Plus a number of papers which admire with the "Btrfs phenomena". Sigh.

Well, let's decide what can we do in current situation.. The first obvious point here is that we *can not* put such file systém to production. Just because it doesn't provide any guarantees for our users regarding disk space utilization.

Ext

http://www.google.cz/#sclient=psy&hl=cs&q=Ext

http://en.wikipedia.org/wiki/Extended_file_system

http://en.wikipedia.org/wiki/Ext2

http://en.wikipedia.org/wiki/Ext3

http://en.wikipedia.org/wiki/Ext4

Linux.

Lustre

http://www.google.cz/#hl=cs&xhr=t&q=Lustre

http://en.wikipedia.org/wiki/Lustre_%28file_system%29

Linux.

Lustre file systems are used in computer clusters ranging from small workgroup clusters to large-scale, multi-site clusters. Fifteen of the top 30 supercomputers in the world use Lustre file systems, including the world's fastest supercomputer.

http://wiki.lustre.org/index.php/Main_Page

http://wiki.lustre.org/index.php/Download

http://wiki.lustre.org/index.php/Lustre_Release_Information#Lustre_Support_Matrix

http://wiki.lustre.org/index.php/Change_Log_2.0

http://wiki.lustre.org/manual/LustreManual20_HTML/index.html

http://wiki.lustre.org/images/6/60/821-2077-10.pdf

http://www.oracle.com/us/products/servers-storage/storage/storage-software/031855.htm

NTFS

http://www.google.cz/#hl=cs&xhr=t&q=NTFS

http://en.wikipedia.org/wiki/NTFS

Windows, GNU/Linux is also supported.

NTFS (New Technology File System) is the standard file system

of Windows NT, including its later versions Windows 2000, Windows XP, Windows Server 2003, Windows Server 2008, Windows Vista, and Windows 7.

http://cs.wikipedia.org/wiki/NTFS

Souborový systém NTFS již od svého počátku velice trpí svou nepříjemnou vlastností, s oblibou fragmentuje soubory. Čtení takto fragmentovaných souborů je pak zbytečně pomalé a čím více je takovýto soubor fragmentován, tím je to horší. Proto se musí oddíly NTFS pravidelně a poměrně často defragmentovat. Tuto skutečnost si mnoho uživatelů neuvědomuje a pak se jen diví, proč jsou u nich diskové operace nepříjemně pomalé.

XFS

http://www.google.cz/#sclient=psy&hl=cs&q=XFS

http://en.wikipedia.org/wiki/XFS

Linux.

Metadata operations in XFS have historically been slower than with other file systems, yielding poor performance with operations such as mass deletions of large numbers of files. This metadata performance issue has been addressed by code created by Red Hat XFS developer Dave Chinner. The feature, a mount option known as "delayed logging", increases the performance of metadata operations by many orders of magnitude, by pushing them almost entirely into memory. The patch was included in the mainline kernel as an experimental feature in 2.6.35, is a stable feature of 2.6.37, and is planned to be the default journal logging method in Linux 2.6.39. Early testing shows performance to now be similar to EXT4 at low thread counts, and superior to EXT4 at high thread counts. See: http://xfs.org/index.php/Improving_Metadata_Performance_By_Reducing_Journal_Overhead and
http://oss.sgi.com/archives/xfs/2010-05/msg00329.html.