Btrfs + flashcache?

I use to have flashcache working a couple weeks ago. I switched to btrfs, but I don't think it works with flashcache because I get badboys like these:
Jul 04 23:55:49 echelon kernel: [<ffffffffa044f4f4>] ? btrfs_merge_delayed_refs+0x214/0x400 [btrfs]
Jul 04 23:55:49 echelon kernel: [<ffffffffa03f486a>] run_clustered_refs+0x68a/0xc40 [btrfs]
Jul 04 23:55:49 echelon kernel: [<ffffffffa03f90d0>] btrfs_run_delayed_refs+0xd0/0x500 [btrfs]
Jul 04 23:55:49 echelon kernel: [<ffffffff81166219>] ? kfree+0x139/0x180
Jul 04 23:55:49 echelon kernel: [<ffffffffa040a987>] __btrfs_end_transaction+0xc7/0x300 [btrfs]
Jul 04 23:55:49 echelon kernel: [<ffffffffa040abd0>] btrfs_end_transaction+0x10/0x20 [btrfs]
Jul 04 23:55:49 echelon kernel: [<ffffffffa0416b34>] btrfs_finish_ordered_io+0x114/0x9c0 [btrfs]
Jul 04 23:55:49 echelon kernel: [<ffffffff8106bf72>] ? del_timer_sync+0x52/0x60
Jul 04 23:55:49 echelon kernel: [<ffffffff814d95fc>] ? schedule_timeout+0x13c/0x2a0
Jul 04 23:55:49 echelon kernel: [<ffffffff8106b260>] ? call_timer_fn+0x1a0/0x1a0
Jul 04 23:55:49 echelon kernel: [<ffffffffa04175d5>] finish_ordered_fn+0x15/0x20 [btrfs]
Jul 04 23:55:49 echelon kernel: [<ffffffffa043730c>] worker_loop+0x12c/0x560 [btrfs]
Jul 04 23:55:49 echelon kernel: [<ffffffff8108e692>] ? default_wake_function+0x12/0x20
Jul 04 23:55:49 echelon kernel: [<ffffffffa04371e0>] ? btrfs_queue_worker+0x300/0x300 [btrfs]
Jul 04 23:55:49 echelon kernel: [<ffffffff8107fb40>] kthread+0xc0/0xe0
Jul 04 23:55:49 echelon kernel: [<ffffffff8107fa80>] ? kthread_create_on_node+0x120/0x120
Jul 04 23:55:49 echelon kernel: [<ffffffff814e552c>] ret_from_fork+0x7c/0xb0
Jul 04 23:55:49 echelon kernel: [<ffffffff8107fa80>] ? kthread_create_on_node+0x120/0x120
Jul 04 23:55:49 echelon kernel: ---[ end trace f480dd00d08c98ac ]---
Jul 04 23:55:49 echelon kernel: BTRFS error (device dm-0) in __btrfs_free_extent:5394: IO failure
Jul 04 23:55:49 echelon kernel: btrfs is forced readonly
Jul 04 23:55:49 echelon kernel: btrfs: run_one_delayed_ref returned -5
Jul 04 23:55:49 echelon kernel: BTRFS error (device dm-0) in btrfs_run_delayed_refs:2565: IO failure
Jul 05 00:02:10 echelon kernel: ------------[ cut here ]------------
Jul 05 00:02:10 echelon kernel: WARNING: at fs/btrfs/super.c:255 __btrfs_abort_transaction+0xcf/0x100 [btrfs]()
Jul 05 00:02:10 echelon kernel: Hardware name: System Product Name
Jul 05 00:02:10 echelon kernel: btrfs: Transaction aborted
Jul 05 00:02:10 echelon kernel: Modules linked in: fuse hidp rfcomm bnep btusb bluetooth rfkill crc16 joydev hid_generic usbhid hid snd_hda_codec_hdmi f
Jul 05 00:02:10 echelon kernel: auth_rpcgss nfs_acl lockd sunrpc microcode asus_atk0110 vboxnetflt(O) vboxdrv(O) btrfs crc32c libcrc32c xor zlib_deflate raid6_pq sr_mod cdrom sd_mod ata_generic pata_acpi ahci firewire_ohci pata
Jul 05 00:02:10 echelon kernel: Pid: 701, comm: btrfs-endio-wri Tainted: P O 3.9.9-1-ck #1
Jul 05 00:02:10 echelon kernel: Call Trace:
Jul 05 00:02:10 echelon kernel: [<ffffffff8105ad90>] warn_slowpath_common+0x70/0xa0
Jul 05 00:02:10 echelon kernel: [<ffffffff8105ae0c>] warn_slowpath_fmt+0x4c/0x60
Jul 05 00:02:10 echelon kernel: [<ffffffffa037b6c9>] ? set_extent_dirty+0x29/0x40 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffffa03350ef>] __btrfs_abort_transaction+0xcf/0x100 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffffa0346375>] __btrfs_free_extent+0x255/0x880 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffffa03a54f4>] ? btrfs_merge_delayed_refs+0x214/0x400 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffffa034a86a>] run_clustered_refs+0x68a/0xc40 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffffa034f0d0>] btrfs_run_delayed_refs+0xd0/0x500 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffffa0360987>] __btrfs_end_transaction+0xc7/0x300 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffffa0360bd0>] btrfs_end_transaction+0x10/0x20 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffffa036cb34>] btrfs_finish_ordered_io+0x114/0x9c0 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffff8106bf72>] ? del_timer_sync+0x52/0x60
Jul 05 00:02:10 echelon kernel: [<ffffffff814d95fc>] ? schedule_timeout+0x13c/0x2a0
Jul 05 00:02:10 echelon kernel: [<ffffffff8106b260>] ? call_timer_fn+0x1a0/0x1a0
Jul 05 00:02:10 echelon kernel: [<ffffffffa036d5d5>] finish_ordered_fn+0x15/0x20 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffffa038d30c>] worker_loop+0x12c/0x560 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffff8108e692>] ? default_wake_function+0x12/0x20
Jul 05 00:02:10 echelon kernel: [<ffffffffa038d1e0>] ? btrfs_queue_worker+0x300/0x300 [btrfs]
Jul 05 00:02:10 echelon kernel: [<ffffffff8107fb40>] kthread+0xc0/0xe0
Jul 05 00:02:10 echelon kernel: [<ffffffff8107fa80>] ? kthread_create_on_node+0x120/0x120
Jul 05 00:02:10 echelon kernel: [<ffffffff814e552c>] ret_from_fork+0x7c/0xb0
Jul 05 00:02:10 echelon kernel: [<ffffffff8107fa80>] ? kthread_create_on_node+0x120/0x120
Jul 05 00:02:10 echelon kernel: ---[ end trace f20aec5935117960 ]---
Jul 05 00:02:10 echelon kernel: BTRFS warning (device dm-0): __btrfs_free_extent:5394: Aborting unused transaction(IO failure).
Jul 05 00:02:10 echelon kernel: btrfs: run_one_delayed_ref returned -5
Jul 05 00:02:10 echelon kernel: BTRFS warning (device dm-0): btrfs_run_delayed_refs:2565: Aborting unused transaction(IO failure).
then the partition gets mounted as RO. Seems bcache works with btrfs though, and I should wait for that

3.10 is in [testing] already.  Also, as of 3.9 there has been the inclusion of dm-cache (which obviouisly utilizes device-mapper).  From the user benchmarks that people have done (though the methods were criticized), they seem to offer some pretty similar performance.

Similar Messages

  • Anyone successfully using Btrfs with dm-cache - any howtos

    Hi
    Does anyone successfully set up dm-cache (or flashcache) on arch? If so is it performing well and stable for you? and how did you set it up (nothing yet on the wiki)?
    thanks in advance

    don't know about dm-cache or flashcache. But at least bcache (which is the one I would choose from the three options if I had a caching drive) shouldn't be stable with btrfs yet according to: https://btrfs.wiki.kernel.org/index.php/Gotchas

  • Slow system with btrfs

    Hi, I have reinstalled system and choose btrfs as system default filesystem. System is overally very slow, maybe this can be also caused by systemd(I had installed archlinux without systemd before), but dont think so.
    It takes about 30s to start any application, openning webpages is slow, downloading....copying between folders is about 5MB/s.
    Can be this caused by btrfs?
    I have noteebook ibm x41, with 1,5ghz cpu, 1,5gb ram.
    Is possible and how to backup data on system disk with btrfs and change filesystem to ext4?
    thanks

    http://arosenfeld.wordpress.com/2010/12 … rom-btrfs/
    https://btrfs.wiki.kernel.org/index.php/Gotchas
    http://www.spinics.net/lists/linux-btrfs/msg14691.html
    http://www.phoronix.com/scan.php?page=a … ions&num=4

  • Maintaining Multiple Arch Installs, Symbolic Links, BTRFS

    Hi again.
    A quick update on my previous post (https://bbs.archlinux.org/viewtopic.php?id=164627). The GRUB problem I was having seems to have stopped. I got a very useful recommendation regarding pacman and XBMC still hasn't been fixed, but is fully usable with the workaround I am using and have just gotten used to it so it's not so much of an issue now. I think now that it should be marked as solved but how do I do that? Was looking around but guess I'm just missing it.
    Anyway now I have a few more issues, or more just questions I guess that I would like help on. Recently I found an old laptop of mine and so to learn more about the Arch system I decided to try and install Arch. Now this laptop is around 5-6 years old and I haven't been able to get WiFi working fully for around 3 years and eventually upgraded 2 years ago to my current laptop. The WiFi was always 'soft-locked' until after about 4-5 restarts (in windows it was fine, but any Linux Distribution post Ubuntu 10.10 had the same issue). The issues still remains in other distributions even after "rfkill all". In Arch ever since running that command (the first boot from install media the issue occurred) this laptops WiFi has been brilliant. I also installed XFCE4 desktop for speed of set up. Now I'm looking to give my laptop to my younger brother before he heads of to University in September until he can afford a new one (or he breaks this one - somehow his laptops never last), as this old one now runs better than it ever did on Windows Vista. My girlfriend is also looking for me to install Linux again on her laptop (we used WUBI in the past) but a full install since her Windows Vista laptop is near unusable in Windows. So far I have got hamachi set up between my new and old laptop, this gives static IP addresses and will allow me to SSH into the laptop for upgrades every so often. I'm planning to do the same for my girlfriends laptop. Would I be stupid to do this via Arch? I've heard it breaks often and I don't want to have to be doing fixes for other peoples laptops as I won't have the time to do so. I guess the reason I want to use Arch is both laptops are quite old and Arch has breathed new life into my old one.
    Secondly I have symbolic links set up in my "Music" and "Video" folders in my home directory to their counterparts in my Windows partition. However my music player "Clementine" doesn't find every album in the music folder. Does symbolic links have "flaws" like this or is it more likely to be my music player behaving badly with symbolic links? I would try another music player but I can't seem to find one which plays FLAC files with external / embedded .cue files so can't really test this out. I can of course link directly to the location but I thought I would ask, just to learn more I guess.
    Now when I installed Arch using BTRFS file system I didn't know that I had to mount using compression immediately, I thought I could add it to the fstab at a later point - obviously not. Now I found a command which will defragment my drive and supposedly compress everything. However this command doesn't make any sense to me and I don't want to just run it without knowing what it does. Could anyone other me any insight into this please? Also does defragmentation after the fstab mount options have been added cause the compression?
    "find -xdev -type f -exec btrfs fi defrag '{}' \;"
    or I also found
    "btrfs filesystem defragment ~/stuff"
    Thanks for any help, I understand this post is quite long so I really appreciate it.
    Last edited by BradPJ (2013-06-13 10:19:47)

    Ah right thanks. I thought as much which is why I asked before I did so, I'll find another distribution. Just where I've been running Arch for a few weeks now and haven't had to do any maintenance at all really, but weren't too sure what I'd be in for in the long-run. I mean to be fair I was always planning on another distribution but as I was playing around at first on the old laptop I ended up with my current way to SSH in and just wondered if it would be viable to do it this way.
    I knew it would be a driver issue at the core of my problem, just weren't too sure if it something else could of been affecting it. Guess I'll try and find another distribution which runs near as new software to Arch as possible as it seems the issue has been fixed in newer versions of the kernel and drivers. Granted the old laptop had been unused for around a year so any update in the last year could of fixed it in this case. Thanks for your input!
    Last edited by BradPJ (2013-06-13 11:35:16)

  • [SOLVED]Arch Linux / UEFI / BTRFS using Grub2 & Windows 8 in a 2nd HDD

    PROBLEM:
    ====================================================================================
    Dear fellas
    I just purchased an new HP TouchSmart 17.3" laptop that comes with Windows 8.1 pro (1 tb HDD + small SSD for cache only ) and still have space for one more HDD or SSD.
    I Google a lot and read a lot but many questions emerged since seems that no one has the same scenario (maybe I pick the wrong choices) like me.
    The problem is.. I didn't wish to re-install Windows 8.1 since it came with from factory.. so I purchased a 750 gb hdd and put it into the free slot to install Arch Linux in a different HDD.
    As I am not familiar with UEFI what I did was to reorder the hdds. I just put the Windows HDD as second disk and the new disk (For Arch Linux) as primary and changed into Bios from UEFI to compatibility mode and installed Arch Linux into the primary one.
    I reaaaally need help to add to grub the correct "path" to Windows 8.1 disk that came with UEFI..
    Anyone could please help me?
    Thanks in advance!
    ====================================================================================
    SOLUTION:
    A huge thanks to @TheSaint and other users for their help and assistance!
    More sources:
    http://www.kossboss.com/linux---arch-in … -grub-boot
    https://www.youtube.com/watch?v=METZCp_JCec#t=146
    https://bbs.archlinux.org/viewtopic.php … 1#p1390741
    Step by Step Summary:
    Use gdisk to create partitions on /dev/sda:
        - 512MB - EF02 type partition (for EFI boot)
        - 690GB - Linux partition for the BTRFS.
    - Create an EF00 (ESP) with 512mb
    - Create a Linux System partition with the rest of space
    Make the FAT 32 system for EFI boot:
    # mkfs.vfat -F32 /dev/sda1
    Make the BTRFS partition. If it complains about existing filesystems just add a "-f":
    # mkfs.btrfs -L arch -f /dev/sda2
    We will make out a root subvolume for sda1, this will be a folder called root located at the root of sda2. The way we will design this is that When the system boots we will not see /root, we will be inside root. Inside root you will have all of your etc,sys,proc,whatever folders etc.
    # mount /dev/sda2 /mnt
    # cd /mnt
    # btrfs subvolume create /mnt/root
    This should show you your root
    # btrfs subvolume list -a /mnt
    Something like this: ID 256 gen 5 top level 5 path root
    # cd /
    # umount /dev/sda2
    Now we will mount sda2 root subvolume as /mnt and we will dump the arch system into there with pacman. We will also enable compress to utilize btrfs compress feature.
    # mount -o defaults,compress=lzo,subvol=root /dev/sda2 /mnt
    NOTE: the command "mount" will not show which subvolume is mounted, to see how subvolumes are mounted you need to look inside proc (cat /proc/self/mountinfo):
    # cat /proc/self/mountinfo | egrep sda2
    The line for the mount of sda2 looks like this:
    43 21 0:34 /root /mnt rw,relatime shared:30 - btrfs /dev/sda2 rw,compress=zlib,ssd,space_cache
    We can see that the subvolume /root is mounted to /mnt from the device /dev/sda3
    Notice how with regular mount command its missing:
    # mount | egrep sda2
    /dev/sda3 on /mnt type btrfs (rw,relatime,compress=zlib,ssd,space_cache)
    Pacman will dump stuff into a boot folder, so we better mount our sda1 EFI boot partition to it. Or else all of the boot stuff will go to sda3 instead of sda1:
    # cd /
    # pacstrap -i /mnt base base-devel
    Let us create the directory and mount the EFI partition
    # cd /mnt
    # mkdir -p /mnt/boot/efi
    # mount /dev/sda1 /mnt/boot/efi
    Let us generate the FSTAB:
    # genfstab -p /mnt >> /mnt/etc/fstab
    Let us chroot into the arch installation:
    # arch-chroot /mnt /bin/bash
    Change password:
    # passwd
    Then pick the right one like this and associate it with a link to /etc/localtime
    # ln -s /usr/share/zoneinfo/US/Pacific /etc/localtime
    Let us generate the initial RAM disk
    # mkinitcpio -p linux
    Let us setup the bootloader (GRUB)
    # pacman -Syu grub efibootmgr
    Let us generate the grub configuration
    # grub-mkconfig -o /boot/grub/grub.cfg
    Let us install grub into the HDD
    # grub-install /dev/sda
    # umount -R /mnt
    # umount /mnt
    # reboot
    From this step you can go straight and forward with the https://wiki.archlinux.org/index.php/Beginners%27_guide
    Last edited by erickwill (2014-11-21 20:41:06)

    TheSaint wrote:As UEFI BIOS is a boot loader itself. You should make on each HDD an ESP.
    When you want to start win8 you go to BIOS and chose its entry, so will do for Arch the same.
    For this way I suggest you write to boot the kernel directly from the BIOS. It just take some reading on this topic
    Other option you set on you second ESP the boot loader of your liking and it will try to find win8 partition.
    Thanks for your reply.
    For the second option, may I use the compatibility mode and install the booloader into the first partition along with Arch?
    Or in case the first option is still the better option, could you pleaaaase give me some directions from the scratch? Do you have Google Hangout?
    Thanks in advance.
    Last edited by erickwill (2014-11-18 19:54:26)

  • [Solved]please help with btrfs read only filesystem

    Yesterday I deleted a snapshot using the command
    sudo btrfs subvolume delete /snapshot-2014-**-**
    in which the snapshot-... is a snapshot I created eariler. But a few hours later I suddenly found that the file system became read-only. I reboot and the message stuck at mounting /dev/sda1 and failed, into an emergency shell, saying something like "no subvolume is find". I tried to mount /dev/sda1 using a rescue usb stick but still failed...
    I have been using the same command to create and delete snapshots for several months without problem, but this time... I don't know what was I doing wrong, maybe I mistyped something this time?
    So... How could I recovery the filesystem? I've searched for a bit but cannot find any satisfied result.
    EDIT: I re-installed the system.
    Last edited by Frefreak (2014-09-27 10:39:45)

    LoBo3268715 wrote:Just gonna take a stab at what you want to do here but I assume you want to switch to the ATI card correct? using open source drivers? Google around for vgaswitcharoo. It can be a royal PITA but I got it working with my intel integrated card and ATI mobility 5650. Also there is a package in the AUR for this. https://aur.archlinux.org/packages.php?ID=51704
    Yes, that's exactly what I'm trying to achieve. Thanks for the advice, I will explore all the possibilities with vga_switcheroo

  • [SOLVED] dual booting windows 7 with btrfs on grub-bios -- core.img

    I am trying to install arch in a dual boot configuration with an existing windows 7 partition. I have everything from the beginner's guide done but the bootloader. When I run grub-install it tells me that core.img is too big.
    Some googling tells me that this is relatively common with btrfs, and it seems the only work around is to switch to gpt mode and use a grub bios partition. But the info I've seen indicates that I need to use MBR mode to dual boot windows.
    Is it safe to do this with windows? Is there another workaround? Or will I have to settle for ext4?
    Last edited by jorenko (2013-06-09 03:53:24)

    Well there's your problem, your first partition starts at sector 63.  With recent versions of windows and fdisk (and every other partitioning tool I can think of off the top of my head) things now align themselves correctly.  Also because there is now GPT, the first partition typically starts later as the GPT partition table will typically sit between the MBR and the first partition. 
    On a MBR partitioned system, grub2 will actually use the first 446 bytes like normal, but will then also use the space that is empty where GPT would sit.  This is why when you have a GPT partitioned system, it will require you to create a 1-2MB grub boot partition, as it needs somewhere else to put its bloat.  GPT actually still uses the MBR section, but simply creates one large partition covering the whole disk.  This is so that tools that are not GPT aware will not think that they have a whole free disk to use as they please.
    For comparison, here is whe I get from fdisk:
    # fdisk -l /dev/sda
    sudo fdisk -l /dev/sda
    [sudo] password for curtisshima:
    WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.
    Disk /dev/sda: 250.1 GB, 250059350016 bytes, 488397168 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk label type: gpt
    # Start End Size Type Name
    1 2048 2099199 1G EFI System EFI System
    2 2099200 252166798 119.2G Linux filesyste arch-btrfs-1
    3 252168192 488397134 112.7G Linux filesyste arch-btrfs-3
    Note that I do use GPT.  But that is not the point here. What I am trying to show is where my first partition starts.  This is also where fdisk will start partitions these days.  This is to ensure compatibility with 4k advanced format disks.
    If you are not dead set on Grub2, you could try using syslinux.  I really like it much better, though if you are booting more than one Linux, you need to either employ chainloading to various partition boot records, or have a shared /boot.  Having a windows partition doens't really matter, as you are simply chainloading to that funky reserved partition anyway. 
    The other option is to use grub-legacy, which can still be found in the AUR.  I actually liked the orginal Grub, as it provided a nice feature set, but was still configurable by hand and it actually fit into the MBR.

  • Trying to understand and learn how to use btrfs

    I have been the past days trying to get my head around how btrfs works
    I have been trying tools like mkinitcpio-btrfs (very poorly documented) and now, if i list the subvolumes i have in certain volume (/) i see i have 4 subvolumes that have been created while playing around.
    If i try to delete them with "sudo btrfs subvolume delete __active", for example, I get a "ERROR: Error accessing '__active'
    What am I doing wrong?
    Also, I cannot get the whole idea about the difference between snapshots and subvolumes, i mean, a snapshot should be a directory that saves the changes made on the fs, s, if you want to roll back those changes, you just have to make btrfs "forget" the changes stored in that directory and move along, but I cannot take the idea of the subvolume thing.....
    As btrfs is quite experimental and the wikis are not noob-proof still, I'd appreciate if someone gave me a hand trying to understand these concept...
    For the moment, I'm just using it on a test computer and on a personal laptop with no fear of data loss,.
    Any help is welcome
    Thanks!
    Last edited by jasonwryan (2013-07-19 23:13:27)

    I honestly think it is probably a better idea to not use mkinitpcio-btrfs.  As mentioned above, it is poorly documented, and for me it has never worked right (if at all).  It is an unofficial AUR package, and unfortunately our wiki still seems to give the false impression that using this package is the way to user btrfs with Arch.
    The way I have my system set up is that in subvolid=0 (the root of the btrfs filesystem) I have a rootfs subvolume and a home subvolume (there are others, but these are what primarily make up my system).  So in my fstab, I basically have two nearly identical lines, but one has no subvol specified and is mounted at /, and the other has 'subvol=home' mounted at /home. 
    So in order to make it so that I can change the root filesystem as I please, instead of having the / fstab entry specify the subvolume, I put it in the kernel command line.  That is, I have 'rootflags=subvol=rootfs' in the kernel command line.  So if I want to change it, I simply change the path to one of the snapshots. 
    Just remember that if you are one who likes a custom kernel, it is likely that you will have to have an initamfs no matter what you compile into your kernel.  For one thing, the kernel has no mechanism for scanning for multiple device btrfs filesystems.  But also, I have read that the kernel itself cannot handle the rootflags kernel command line argument.
    Oracle Linux does something interesting with their default setup.  They are not a rolling release, so this probably wouldn't work so well in Arch Linux, but they actually install the root filesystem (I think it is actually done to subvolid=0) and then after installation of the packages, a snapshot of the root filesystem is made, and the system is setup to boot off of that snapshot.  So it is almost like having an overlayfs on openwrt.  There is always a copy of the original system, and and changes that are being made are being done "on top" of the original.  So in the event of an emergency, yo can always get back to the original working state.
    If you put your root filesystem on something other than the root of the btrfs filesystem (which you should, as it makes the whole setup much more flexible), then you should also set up a mountpoint somewhere to give administrator access to the filesystem from subvolid=0.  For example, I have an autofs mountpoint at /var/lib/btrfs-root.   chose that spot because /var/lib is where devtools puts the clean chroot.  So it seemed as reasonable a place as any.
    You should go to the btrfs wiki, and peruse through the stuff there... not our wiki, but the actual btrfs one, as our wiki is pretty sparse.  There is not all that much content there (not like the Arch wiki), but it does cover the features pretty well.  I mean, there is certainly quite a lot for being information on only a single filesystem, but it shouldn't take you too long to get through it.  There are a few links to articles about midway down the front page.  What really gave me a better grasp of getting started with btrfs were the ones titled "How I Got Started with the Btrfs Filesystem for Oracle Linux" and "How I Use the Advanced Capabilities of Btrfs".

  • Trying to understand how BTRFS works

    Hello,
    I read most if not all of the articles, presentations and podcasts about Btrfs at https://btrfs.wiki.kernel.org/index.php/Main_Page and other sites. All is very nice. I had to read some of the information a couple of times though to more or less understand and I'm not sure if I really do.
    I'm trying to explain BTRFS in my own words, but wonder if it is correct. What about the following:
    The storage space of a BtrFS file system is comprised of file data blocks and metadata blocks. The BtrFS metadata is organized in a B-tree fashion and describes the version and location of data on disk. The BtrFS file system uses a copy-on-write (CoW) storage strategy. An existing file under BtrFS is never initially overwritten. Instead, when modifying a file, data blocks are copied, modified and written anywhere on disk according to metadata to prevent overwriting of data required by snapshots. When a file under Btrfs is modified, data is not necessarily overwritten. Modified data blocks of files can be written anywhere on disk according to metadata and requirement to maintain existing snapshots. A snapshot does initially not impose additional storage space until data blocks are modified.
    Is this correct? And if not, where am I wrong please?
    Thanks!
    Edited by: Dude on Oct 16, 2012 10:38 AM

    Dude wrote:
    Thanks for the feedback. Actually that's one of the parts I'm not sure if I understand it correctly. It is my understanding that Btrfs works on the data block level, which I think is the smallest amount of data that can be allocated. Similar to Oracle database. So in order to modify data, the whole block needs to be written. For example, let's say I change a the word "teh" to "the" in a file. Doesn't it have to read the whole block, modify the content, and check the metadata before writing the complete block?Yes. But it does tight packing, so it probably wouldn't write an entire block out for "the" -- it would just squash it into one of the leaves of the b-trees.

  • Trying to understand BtrFS snapshot feature

    I'm trying to understand how the copy-on-write and Btrfs snapshot works.
    Following simple test:
    <pre>
    # cd /
    # touch testfile
    # ls --full-time testfile
    -rw-r--r-- 1 root root 0 2012-10-15 12:04:43.629620401 +0200 testfile
    Test 1:
    # btrfs subvol snapshot / /snap1
    # touch testfile
    # ls --full-time testfile /snap1/testfile
    -rw-r--r-- 1 root root 0 2012-10-15 12:04:43.629620401 +0200 /snap1/testfile
    -rw-r--r-- 1 root root 0 2012-10-15 12:07:38.348932127 +0200 testfile
    Test 2::
    # btrfs subvol snapshot / /snap2
    # touch testfile
    # ls --full-time testfile /snap1/testfile /snap2/testfile
    -rw-r--r-- 1 root root 0 2012-10-15 12:04:43.629620401 +0200 /snap1/testfile
    -rw-r--r-- 1 root root 0 2012-10-15 12:07:38.348932127 +0200 /snap2/testfile
    -rw-r--r-- 1 root root 0 2012-10-15 12:09:21.769606369 +0200 testfile
    </pre>
    According to the above tests I'm concluding/questioning the following:
    1) Btrfs determines which snapshot maintains a logical copy and physically copies the file to the appropriate snapshot before it is modified.
    a) Does it copy the complete file or work on the block level?
    b) What happens if the file is very large, e.g. 100 GB and there is not enough space on disk to copy the file to the snapshot directory?
    c) Doesn't it have a huge negative impact on performance when a file needs to be copied before it can be altered?

    Hi, thanks for the answers!
    I guess calling it "logical copy" was a bad choice. Would calling the initial snapshot a "hard link of a file system" be more appropriate?
    Ok, so BTRFS works on the block level. I've done some tests and can confirm what you said (see below)
    I find it interesting that although the snapshot maintains the "hard link" to the original copy - I guess "before block image" (?) - there really is no negative performance impact.
    How does this work? Perhaps it is not overwriting the existing file, but rather creating a new file? So the snapshot still has the "hard link" to the original file, hence nothing changed for the snapshot? Simply a new file was created, and that's showing in the current file system?
    It actually reminds me of the old VMS ODS filesystem, which used file versioning by adding a simicolon, e.g. text.txt;1. When modifying the file the result would be text.txt;2 and so on. When listing or using the file without versions, it would simply show and use the last version. You could purge old version if necessary. The file system was actually structured by records (RMS), similar like a database.
    <pre>
    [root@vm004 /]# # df -h /
    Filesystem Size Used Avail Use% Mounted on
    /dev/sda3 16G 2.3G 12G 17% /
    # time dd if=/dev/zero of=/testfile bs=8k count=1M
    1048576+0 records in
    1048576+0 records out
    8589934592 bytes (8.6 GB) copied, 45.3253 s, 190 MB/s
    Let's create a snapshot and overwrite the testfile
    # btrfs subvolume snapshot / /snap1
    # time dd if=/dev/zero of=/testfile bs=8k count=1M
    dd: writing `/testfile': No space left on device
    491105+0 records in
    491104+0 records out
    4023123968 bytes (4.0 GB) copied, 21.2399 s, 189 MB/s
    real     0m21.613s
    user     0m0.021s
    sys     0m3.325s
    <pre>
    So obviously the there is not enough space to maintain the original file and the snapshot file.
    Since I'm creating a complete new file, I guess that's to be expected.
    Let's try with a smaller file, and also check performance:
    <pre>
    # btrfs subvol delete /snap1
    Delete subvolume '//snap1'
    # time dd if=/dev/zero of=/testfile bs=8k count=500k
    512000+0 records in
    512000+0 records out
    4194304000 bytes (4.2 GB) copied, 21.7176 s, 193 MB/s
    real     0m21.726s
    user     0m0.024s
    sys     0m2.977s
    # time echo "This is a test to test the test" >> /testfile
    real     0m0.000s
    user     0m0.000s
    sys     0m0.000s
    # btrfs subvol snapshot / /snap1
    Create a snapshot of '/' in '//snap1'
    # df -k /
    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/sda3 16611328 6505736 8221432 45% /
    # time echo "This is a test to test the test" >> /testfile
    real     0m0.000s
    user     0m0.000s
    sys     0m0.000s
    # df -k /
    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/sda3 16611328 6505780 8221428 45% /
    # btrfs subvol delete /snap1
    Delete subvolume '//snap1'
    # df -k /
    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/sda3 16611328 6505740 8221428 45% /
    The snapshot occupied 40k
    # btrfs subvol snapshot / /snap1
    Create a snapshot of '/' in '//snap1'
    # time dd if=/dev/zero of=/testfile bs=8k count=500k
    512000+0 records in
    512000+0 records out
    4194304000 bytes (4.2 GB) copied, 21.3818 s, 196 MB/s
    real     0m21.754s
    user     0m0.019s
    sys     0m3.322s
    # df -k /
    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/sda3 16611328 10612756 4125428 73% /
    There was no performance impact, although the space occupied doubled.
    </pre>

  • [SOLVED] Install to BtrFS already existing (but empty) subvolume

    I currently have a Btrfs filesystem made up of /dev/sda5 and /dev/sda6. On that filesystem, there is an Ubuntu 11.10 install in the @oneiric_main subvolume and a data subvolume (with Ubuntu /home) in the @Data subvolume. Grub2 is installed in the @oneiric_main subvolume and works fine.
    I would like to install arch to a new subvolume on the existing BtrFS filesystem. I would probably make a subvolume named @arch_2011.08 to install it on. (@arch is taken--I already tried to do this myself, installing using pacman -r <subvol_mount_point> -Sy base --ignore grub. Didn't work, has no initrd, kernel panics with VFS not syncing error) I would like to then boot Arch with Ubuntu's Grub2 since that install is working off BtrFS. How do I go about doing this?
    On the install that failed, I generally tried to follow https://wiki.archlinux.org/index.php/In … Btrfs_root, but I didn't really need/want automatic snapshots, so I didn't think I wanted to install mkinitcpio-btrfs. For it my Grub2 entry is
    menuentry "Arch (on /dev/sda6)" --class gnu-linux --class gnu --class os {
    insmod part_msdos
    insmod btrfs
    set root='(hd0,msdos6)'
    search --no-floppy --fs-uuid --set=root d551dbac-3c7f-4f8f-a0d4-2d1f242cc42f
    linux /@arch/boot/vmlinuz-linux root=/dev/disk/by-uuid/d551dbac-3c7f-4f8f-a0d4-2d1f242cc42f rootflags=subvol=@arch ro
    I've tried to chroot to the Arch install from Ubuntu and run mkinitcpio, but that gave an error that some directory was not valid. The directory it complained about had the Ubuntu kernel version in its name, so I figured I was doing something majorly wrong. Edit:I suppose it would help if I updated the /etc/fstab. I did, but I still get the same error. The first line is "Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)"
    I'm not sure if I should try to fix the first install or just start again. Either way would get the goal:Arch installed on the BtrFS subvolume, booted with Ubuntu's Grub2.
    Last edited by Azendale (2011-12-17 22:34:16)

    @Teekay:I did mount /proc, /sys and /dev into the @arch subvolume before chrooting.
    Here's what happens when I try to chroot into the Arch install to make an initrd:
    [root@dynl:~]# mount -o subvol=@arch /dev/sda5 /mnt
    [root@dynl:~]# cd /mnt
    [root@dynl:/mnt]# mount -o bind /dev dev
    [root@dynl:/mnt]# mount -o bind /sys sys
    [root@dynl:/mnt]# mount -o bind /proc proc
    [root@dynl:/mnt]# chroot .
    chroot: failed to run command `/bin/zsh': No such file or directory
    [root@dynl:/mnt]# chroot . bash
    bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
    [root@dynl /]# ls
    bin dev home lib64 mnt proc run srv tmp var
    boot etc lib media opt root sbin sys usr
    [root@dynl /]# mount
    /dev/sda5 on / type btrfs (rw,relatime)
    udev on /dev type devtmpfs (rw,relatime,size=693104k,nr_inodes=173276,mode=755)
    sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
    proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
    [root@dynl /]# mkinitcpio
    /bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
    ==> ERROR: '/lib/modules/3.1.0-030100rc10-generic' is not a valid kernel module directory
    Update: I've made progress. Instead of chrooting from Ubuntu, I chooted from the arch installation live image. I was able to run mkinitcpio -p linux. I added an initrd line to grub2:
    menuentry "Arch (on /dev/sda6)" --class gnu-linux --class gnu --class os {
    insmod part_msdos
    insmod btrfs
    set root='(hd0,msdos6)'
    search --no-floppy --fs-uuid --set=root d551dbac-3c7f-4f8f-a0d4-2d1f242cc42f
    linux /@arch/boot/vmlinuz-linux root=/dev/disk/by-uuid/d551dbac-3c7f-4f8f-a0d4-2d1f242cc42f rootflags=subvol=@arch ro
    initrd /@arch/boot/initramfs-linux.img
    The new error is :
    Decompressing Linux... Parsing ELF... done.
    Booting the kernel.
    :: Starting udevd...
    done.
    :: Running Hook [udev]
    :: Triggering uevents...done.
    Waiting 10 seconds for device /dev/sda6 ...
    mount: mounting dev/sda6 on /new_root failed: Invalid argument
    ERROR: Failed to mount the real root device.
    bailing out, you are on your own. Good luck.
    sh: can't access tty; job control turned off
    [ramfs /]#
    Also, the Arch /etc/fstab is:
    # /etc/fstab: static file system information
    # <file system> <dir> <type> <options> <dump> <pass>
    tmpfs /tmp tmpfs nodev,nosuid 0 0
    UUID=d551dbac-3c7f-4f8f-a0d4-2d1f242cc42f / btrfs defaults,subvol=@arch 0 0
    Last edited by Azendale (2011-11-05 06:43:49)

  • Fresh btrfs root partition cannot boot/unknown filesystem [solved]

    Just reformatted my SSD with 3 partitions.
    1) EFI
    2) /boot (ext4)
    3) / (btrfs)
    I copied over my systefrom tars, adjusted the new fstab to use btrfs, I rebuilt my kernel images, and rebuilt grub.cfg in the chroot.  When I reboot, I get:
    Welcome to grub!
    error: unknown filesystem
    Entering rescue mode...
    grub rescue>
    Per the wiki, there should be no special setup steps to boot to a btrfs root with grub.  When did I f-up?
    /etc/fstab
    LABEL=arch64 / btrfs defaults,discard,noatime,ssd,compress=lzo 0 0
    LABEL=boot /boot ext4 defaults,discard,noatime 0 2
    Excuse the short post, typing in a phone.
    Last edited by graysky (2013-07-29 01:24:47)

    Yes, it is dropped to a GRUB rescue shell.  I can boot if I manually type the following but WTF?!  Why isn't grub doing this by itself?
    grub rescue> set prefix=(hd0,2)/grub
    grub rescue> insmod (hd0,2)/grub/x86_64-efi/linux.mod
    grub rescue> set root=(hd0,2)
    grub rescue> linux /vmlinuz-linux-ck root=/dev/sda3
    grub rescue> initrd /initramfs-linux-ck.img
    grub rescue> boot
    Link to my autogenerated /boot/grub/grub.cfg
    Last edited by graysky (2013-07-29 00:28:58)

  • Can btrfs(2.6.31) file system be used during an install

    Hi,
    I would like to use btrfs for my root and home partition during an initial installation, but I notice that the btrfs in kernel 2.6.30 (what's on the CD) is old and has had the file system
    format changed as of 2.6.31, so is not backward compatible.
    Is there any way to install using the latest btrfs from 2.6.31 (or 2.6.32 if it is out ..)  .... or am I just too beeding edge on this ..
    Cheers,
    Bernie

    Cool, I am not the only one as it seems
    I might be wrong but doesn't btrfs handle file system upgrades? But even then it doesn't seem to be possible to setup the root filesystem for btrfs currently. Because the setup enviroment lacks the proper tools, like mkfs.brtfs.
    Maybe stability is an issue here. Some user might get very angry if their data gets lost After all btrfs is still under development.

  • Btrfs May Be The Default File-System For Fedora 16

    Having a somewhat odd interest in different types of filesystems, I'd thought I'd share this snippet I wondered across.
    Brought up on the Fedora development list are the plans for Btrfs in Fedora, which provides a target of Fedora 16 when EXT4 will be replaced by Btrfs as the default Linux file-system on new installations.
    Fedora was one of the first distributions to deploy support for installing the Linux distribution to a Btrfs root file-system, while the Moblin/MeeGo camp has already turned to it as the default file-system, and now Fedora may finally have everything ready and are comfortable with the state of this Oracle-sponsored file-system.
    With the intended Fedora deployment of Btrfs, they will also switch from using LVM as the disk's volume manager to instead relying upon the volume management abilities built into Fedora itself. Fedora / Red Hat developers have previously already worked on taking advantage of other Btrfs features, like snapshots, to provide system roll-back support by creating a copy-on-write snapshot prior to each yum/RPM package transaction.
    In order to meet the Fedora 16 Btrfs target, Btrfs support needs to be merged into Fedora's GRUB package (or a separate non-Btrfs /boot file-system must be used), Red Hat's Anaconda must be hooked in to properly utilize Btrfs and its volume management features, and the Fedora LiveCD must be re-worked a bit to handle Btrfs. Additionally, the fsck support for Btrfs must be completed. Oracle's Chris Mason is nearly (circa 90%) complete with this task.
    Before Fedora 16 arrives we still have Fedora 15 to worry about, which will continue to offer Btrfs as a file-system option while EXT4 remains the default.
    Canonical is also evaluating when to switch to the Btrfs file-system and we may see that move similarly happen for Ubuntu 11.10.
    This would seem to imply that fedora is giving btrfs the big thumbs up! They currently release twice a year, and  fedora 15 is (I believe) Due out this May, so it wouldnt be too wrong to surmise that come Christmas btrfs would be the standard install option. However, I have always thought of fedora as being a little bit of a testing bed for red hat enterprise, so reading between the lines, I am thinking they want to release btrfs to the wild, and get the bugs sqwished so it can be incorporated into red hat, which is a good thing all round, for all concerned.
    Quote blatantly ripped from phoronix.
    wikipedia btrfs
    kernel.org btrfs

    There are, as I understand it, three big issues keeping Btrfs from hitting its final "stable" release:
    1) No true fsck (as of November, a fix is due "any time now")
    2) The logic used for subvolumes (a sort of "virtual LVM" setup) in Btrfs is wholly different from that used in Ext*, JFS, NTFS, HFS, etc; because of this, there isn't yet a single script/program that will accurately display space consumption.  Instead, the user needs to run a couple different commands and extrapolate.  A little annoying, but it doesn't affect stability.
    3) A bug in the COW aspect of the defrag command which, if run with an incorrect flag, can cause a doubling in the amount of metadata on the filesystem (and thus the amount of space it consumes).  Again, it won't wreck anything, but would make management tedious if one needs to restore a snapshot to get rid of extra cruft.
    These are all mentioned in the FAQ's on the Btrfs wiki, though the wiki doesn't give priority to any of them.  I'm guessing that, if Fedora 16 will be using this later in the year, the Oracle folks have told the Red Hat folks that they're planning on plowing through these issues very soon.
    Last edited by ANOKNUSA (2011-03-01 14:55:46)

  • Clarification on Btrfs Subvolumes

    Howdy-ha, folks.  So, I just got myself a new Western Digital Advance Format HDD for my laptop.  While I first considered cloning my current drive, I decided instead to just reinstall by backing up/restoring the pacman cache, first formatting the drive as Btrfs.  I've hit a snag, though, because despite reading through the Brtfs wiki FAQ, Use Cases and Getting Started entries word-for-word (twice, even), the Arch wiki entry on Btrfs and searching the forums I can't quite wrap my head around subvolumes.  I understand that the general idea is to do away with multiple, partially-full partitions, but I can't figure out how subvolumes are mounted and used as an analog to partitions.
    I plan to use Archboot to install, and up to this point have had separate partitions for /root and /home.  In order to mimic this with Btrfs, would I simply create two subvolumes at installation time--one for root to allow for snapshots, and one for my Home folder?  Would these actually be mounted in theoretically the same way as separate partitions?  I like to tinker, and 'till now have just used clonezilla to back-up my /root partition if something went wrong, leaving /home untouched.  Sorry, but I can't figure out how this might be done, as the wiki simply explains how to create a new subvolume in a certain location and mount it manually (since it isn't a new filesystem, it doesn't need an fstab entry); and how to delete a subvolume.  It also seems that the examples used in the Btrfs wiki are for an external device, which just seems to obfuscate things.  I'd really appreciate the help of someone experienced in this or a point in the right direction, though I suppose I could also just create two separate Btrfs partitions if I can't figure it out.  Thanks.

    So as I write this, I think I may have found a solution to the first question, and have a functional system.  I used Archboot to install this setup:
    $ btrfs sub li /
    ID 256 top level 5 path root
    ID 257 top level 5 path home
    Which, when viewed from a LiveCD, looks essentially like this:
    __actual root__
    |__subvbolume "root"
    |__subvolume "home"
    Each of these is mounted via fstab in the respective part of the file system--or at least it seems so:
    #/etc/fstab: static file system information
    # <file system> <dir> <type> <options> <dump> <pass>
    devpts /dev/pts devpts defaults 0 0
    shm /dev/shm tmpfs nodev,nosuid 0 0
    # DEVICE DETAILS: /dev/sda1 UUID=28a77746-673e-482c-b037-3eea590c0b14 LABEL=Boot
    # DEVICE DETAILS: /dev/sda2 UUID=7caf3bbf-0d2c-4961-8693-962ebc365f20 LABEL=Arch
    # DEVICE DETAILS: /dev/sda2 UUID=7caf3bbf-0d2c-4961-8693-962ebc365f20 LABEL=Arch
    # DEVICE DETAILS: /dev/sda3 UUID=f6e8ba7e-0cae-429d-9a2b-e4bdf5d8be7d LABEL=Swap
    LABEL=Arch / btrfs defaults,subvol=root,compress 0 1
    LABEL=Arch /home btrfs defaults,subvol=home,compress 0 1
    LABEL=Boot /boot ext2 defaults 0 1
    LABEL=Swap swap swap defaults 0 0
    This results in a seemingly ordinary filesystem when I'm logged into my desktop:
    ├── bin
    ├── boot
    ├── dev
    ├── etc
    ├── home
    ├── lib
    ├── lib64
    ├── media
    ├── mnt
    ├── opt
    ├── proc
    ├── root
    ├── sbin
    ├── srv
    ├── sys
    ├── tmp
    ├── usr
    └── var
    /home
    └── anoknusa
    ├── ABS
    ├── Desktop
    ├── Documents
    ├── Downloads
    ├── Music
    ├── Pictures
    ├── pkglist
    ├── Stuff
    ├── Torrents
    └── Videos
    So, at this juncture I have what appear to be two separate snapshots mounted at separate points in order to create a single filesystem, just as I would had I mounted to separate partitions, and I'm left only wondering about the second question.
    I'm still not entirely clear whether this will give me the desired effect.  Each subvolume rests at the same level of the fs tree; they're side-by-side, rather than having the "home" subvolume under the "root" subvolume.  Again, what I'm hoping for is the ability to take snapshots of "root" and "home" separately--as one would with separate partitions--so that taking a snapshot of "root" won't end up with my entire drive under it, and I can tinker without worrying about whether or not I'll have to spend two hours restoring a snapshot every few days.  I'm not sure if this would work while both snapshots are mounted, so I'm planning on testing this once my backups are done copying to my ~/  folder (I'm really no good when it comes to visualizing this sort of thing, and need to either see it in action or be told that it's gonna work by someone who's done this before). Hopefully, this works out as planned.  Any more advice from anyone would be greatly appreciated; thanks again.

Maybe you are looking for