filedb

FileDB - A C# database to store files
FileDB is a free, fast, lightweight C# (v3.5) DLL project to store, retrive and delete files using a single file container on disk. Ideal for store small, medium or big files without databases and keep organized on a single disk file.


Update 2015-01-24

Try LiteDB, my new NoSql database with file stream support for C#. http://www.litedb.org

FileDB - Project Description

FileDB is a free, fast, lightweight C# (v3.5) DLL project to store, retrieve and delete files using a single archive file as a container on disk. It's ideal for storing files (all kind, all sizes) without databases and keeping them organized on a single disk file.

News

  • First stable version was launched
  • Improvements on concurrent write access
  • New method: Export - Export all files to a directory

Let's see how to use FileDB with static helper methods.

    using Numeria.IO;

    var pathDB = @"C:\Temp\MyDB.fdb";

    // Creating an empty FileDB archive
FileDB.CreateEmptyFile(pathDB); // Store file from input stream
var info = FileDB.Store(pathDB, "MyFileName.jpg", inputStream);
// -or- store directly from the file itself
var info = FileDB.Store(pathDB, @"C:\Temp\MyPhoto.jpg"); // The 'info' variable returned contains information about your file (generated GUID, filename, file-length, mime-type)
var fileGuid = info.ID; // Reading file inside FileDB and writing it on an output stream (also available to write it directly to a file)
var info = FileDB.Read(pathDB, fileGuid, outputStream); // Deleting a file
var ok = FileDB.Delete(pathDB, fileGuid);

FileDB Methods

  • CreateEmptyFile - Create an empty data file archive
  • Store - Store from file/stream to the data file
  • Read - Search a fileID and restore it to output file/stream
  • Delete - Delete a file
  • ListFiles - List all files inside the archive
  • Shrink - Reorganize archive removing unused disk space
  • Export - Export files inside archive to a directory

All operations have a static method helper or can be used from a FileDB instance.

ASP.NET MVC Example

Below is a basic example to store/retrieve/delete information in a ASP.NET MVC Controller

    private string pathDB = @"C:\Temp\MvcDemo.fdb";

    // Uploading a file
[HttpPost]
public ActionResult Upload()
{
HttpPostedFileBase file = Request.Files[0] as HttpPostedFileBase; if (file.ContentLength > 0)
FileDB.Store(pathDB, file.FileName, file.InputStream); return RedirectToAction("Index");
} // Download
[HttpGet]
public ActionResult Download(string id)
{
// Using a classic way to download a file, instead of FileContentResult mode.
// Optimizing for big files. Your webserver will not consume CPU/Memory to download this file (even very large files) using (var db = new FileDB(pathDB, FileAccess.Read))
{
var info = db.Search(Guid.Parse(id)); Response.Buffer = false;
Response.BufferOutput = false;
Response.ContentType = info.MimeType;
Response.AppendHeader("Content-Length", info.FileLength.ToString());
Response.AppendHeader("content-disposition", "attachment; filename=" + info.FileName); db.Read(info.ID, Response.OutputStream); return new EmptyResult();
}
}

Why?

Well, all web developers already had this problem: "I need to store same user files (photos, documents, ...) and I need to save them on my web server. But where? File system or database?" (See some discussion in http://stackoverflow.com/questions/3748/storing-images-in-db-yea-or-nay#3756)
The problem is: the database was not designed to store files. ADO.NET doesn't have a good method to work with streams (only byte[]). For small files (less then 100K) it works very nice. But what happens with a 10Mb file? 100Mb? It's terrible! The solution: work on filesystem!
The filesystem has a problem too: what if my user wants to upload 300 files? And what if I have 2000 users? You will have thousands of files to backup and manage, and this will be a pain.
My idea: a single archive (or a single archive per user) to store only user uploaded files. I use the SQL Server database to store the ID file reference (GUID) and FileDB does the rest of the job, storing and managing the files and bytes.

How FileDB works?

FileDB was designed to be fast, very simple, file-based and works with all file sizes. FileDB doesn't consume lots of memory because it works only with streams (no byte[]). The data structure is built with two kinds of pages: IndexPage and DataPage.
- The IndexPage stores information about the file descriptor and it's organized in a binary tree structure.
- The DataPage stores the file bytes.
Both pages have 4096 bytes (plus 100 bytes to file header). Each page has its own header to store information about the data inside the page.
You can delete a file inside the archive and the empty data pages will be used on next insert. Or you can shrink database to get your non-used bytes.
FileDB caches (in memory) only index pages, to be faster on a second search.

Limitations

I've made many tests to check performance and limitations with all file sizes. To protect against many clients changing data on same archive, FileDB uses read share mode. This way many users can search/read simultaneously but only one user can write (store/delete) at a time. FileDB class also implements IDisposable so you can use it inside a using code.

    using(var db = new FileDB(pathDB, FileAccess.ReadWrite))
{
db.Store(@"C:\Temp\MyPhoto.jpg");
}

The data size limitations are based on .NET MaxValue constants. FileDB works with UInt32 (4 bytes unsigned), which limits each file to 4GB and the database to 16TB (4096 Pages * UInt32.MaxValue).

Future

  1. I intend to create a windows client application to explore the contents of a FileDB database. It will be a useful tool to manage files inside a database.
  2. Create a compressed data page in which data can be stored in a zip mode.
  3. Any other suggestions?

Contribute

Please, feel free to help and contribute with this project adding you comments, issues or bugs found.

FileDb的更多相关文章

  1. mysql_建立索引的优缺点 #转自Starzm#

    建立索引的优缺点: 为什么要创建索引呢? 这是因为,创建索引可以大大提高系统的性能.         第一.通过创建唯一性索引,可以保证数据库表中每一行数据的唯一性.         第二.可以大大加 ...

  2. python基础-装饰器

    一.什么是装饰器 装饰器本质就是函数,功能是为其他函数附加功能 二.装饰器遵循的原则 1.不修改被修饰函数的源代码 2.不修改被修饰函数的调用方式 三.实现装饰器的知识储备 装饰器=高阶函数+函数嵌套 ...

  3. socketserver模块写的一个简单ftp程序

    一坨需求... 用户加密认证 允许同时多用户登录 每个用户有自己的家目录 ,且只能访问自己的家目录 对用户进行磁盘配额,每个用户的可用空间不同 允许用户在ftp server上随意切换目录 (cd) ...

  4. 浅析MongoDB数据库的海量数据存储应用

    [摘要]当今已进入大数据时代,特别是大规模互联网web2.0应用不断发展及云计算所需要的海量存储和海量计算发展,传统的关系型数据库已无法满足这方面的需求.随着NoSQL数据库的不断发展和成熟,可以较好 ...

  5. 【转】MySQL外键约束On Delete、On Update各取值的含义

    转载地址:http://hi.baidu.com/jxqlovejava/item/3d2cc5b5d689917c244b0920 ‍ 先看On Delete属性,可能取值如上图为:No Actio ...

  6. 在peopletools里面测试文件上传

    Using the PeopleTools Test Utilities Page Select selectPeopleTools, then selectUtilities, then selec ...

  7. MySQL外键约束On Delete、On Update各取值的含义

    主键.外键和索引的区别?   主键 外键 索引 定义: 唯一标识一条记录,不能有重复的,不允许为空 表的外键是另一表的主键, 外键可以有重复的, 可以是空值 主索引(由关键字PRIMARY定义的索引) ...

  8. Java基于文件的对象存储

    工作中经常需要处理对象的处理,有的时候还需要将对象保存到文件中做持久化. 特别是当不能使用数据库的时候就特别需要一个简单的对象集合的增删改查操作, 于是就有了下面这个文件DB的工具类 package ...

  9. 利用whoosh对mongoDB的中文文档建立全文检索

    1.建立索引 #coding=utf-8 from __future__ import unicode_literals __author__ = 'zh' import sys,os from wh ...

随机推荐

  1. MFC中创建自定义消息

    消息映射.循环机制是Windows程序运行的基本方式.VC++ MFC 中有许多现成的消息句柄,可当我们需要完成其它的任务,需要自定义消息,就遇到了一些困难.在MFC ClassWizard中不允许添 ...

  2. 模块化 Sea.js(CMD)规范 RequireJS(AMD)规范 的用法

    插入第三方库AMD CMD都 一样  如:JQ(再JQ源码里修改) 使用seajs的步骤 1.HTML里引入seajs <script src="./lib/sea.js"& ...

  3. RAID阵列盘有一块状态变为外来处理方法

    感谢: https://blog.csdn.net/cmzsteven/article/details/63680933

  4. Spring 基础知识(一)基本概念 DI、IOC、AOP

    DI(依赖注入) 和IOC(控制反转)都是一种设计思想,要理解他们,让我们从coding中的一些痛点入手. 依赖注入 Dependency Injection : 如果A类要使用B类的一个方法,首先必 ...

  5. IT技术

    一.通信网络 TCP/IP协议 路由交换技术 二.编程语言 C/C++ python JAVA 三.数据库 关系型数据库 (1)MySQL  MySQL学习笔记一 MySQL学习笔记二 2.  非关系 ...

  6. java基础知识—类和对象

    1.对象的特征---类的属性 每个对象的每个属性都有特定的值 对象的操作---类的方法 2.封装 对象同时具有属性和方法两项属性. 对象的属性和方法同时被封装在一起,共同体现事物的特性,二者相辅相成, ...

  7. Node.js和html数据交互(一) form表单

    一.form表单提交数据 数据流向:前端 - > 后端 1.get方法 (action是提交路径,method是提交方法,get方法可以不写) 前端: <form action=" ...

  8. 2018-2019-2 网络对抗技术 20165326 Exp3 免杀原理与实践

    免杀原理与实践 目录 知识点问答 实践内容 遇到的问题 心得体会 知识点 meterpreter免杀 基础问题回答 杀软是如何检测出恶意代码的? 特征码(基于签名):模式匹配,比对特征码库 启发式:通 ...

  9. Centos6.5 安装 RabbitMQ 3.7.11

    RabbitMQ是一个开源的AMQP实现,服务器端用Erlang语言编写,支持多种客户端,如:Python.Ruby..NET.Java.JMS.C.PHP.ActionScript.XMPP.STO ...

  10. OpenGL之纹理贴图(Texture)

    学习自: https://learnopengl-cn.github.io/01%20Getting%20started/06%20Textures/ 先上一波效果图: 实际上就是:画了一个矩形,然后 ...