Compare commits

...

159 Commits

Author SHA1 Message Date
Alex Ling
ac3df03d88 Show page counts on chapter cards 2020-04-02 05:44:29 +00:00
Alex Ling
7c9728683c On the title page, hide progress label of nested titles 2020-04-02 00:16:19 +00:00
Alex Ling
d921d04abf Bump version to v0.2.4 2020-04-01 23:32:16 +00:00
Alex Ling
5400c8c8ef Fix a UI bug that shows "resume download" button on download manager even when the downloading process is not paused 2020-04-01 23:21:32 +00:00
Alex Ling
58e96cd4fe Watch the title element size for change 2020-04-01 06:13:03 +00:00
Alex Ling
aa09f3a86f Only show tooltips for truncated titles 2020-04-01 05:59:46 +00:00
Alex Ling
a5daded453 Fix the width and height of cover images (#23) 2020-04-01 04:51:57 +00:00
Alex Ling
4968cb8e18 Add tooltips to show un-truncated titles 2020-04-01 04:49:53 +00:00
Alex Ling
27c6e02da8 Run the truncate function after DOM is ready 2020-04-01 04:48:53 +00:00
Alex Ling
68d1b55aea Limit title text height in CSS 2020-04-01 04:47:55 +00:00
Alex Ling
32dc3e84b9 Lazy load images in library/title page to improve page load time 2020-03-31 08:44:07 +00:00
Alex Ling
460fcdf2f5 Limit the number of lines to display in card titles 2020-03-30 20:36:27 +00:00
Alex Ling
c6369f9f26 Prevent flash of white in cards 2020-03-30 20:35:30 +00:00
Alex Ling
aa147602fc Bump version number 0.2.2 -> 0.2.3 2020-03-27 05:00:14 +00:00
Alex Ling
d58c83fbd8 Use BigInt when sorting filenames (#22) 2020-03-27 04:45:03 +00:00
Alex Ling
1a0c3d81ce Add Patreon 2020-03-21 05:18:53 +00:00
Alex Ling
33c61fd8c1 Add build badge 2020-03-19 16:04:06 -04:00
Alex Ling
6eba3fe351 Create build.yml 2020-03-19 19:58:59 +00:00
Alex Ling
da2708abe5 Put mango binary in / instead of /root/Mango/ 2020-03-19 18:17:26 +00:00
Alex Ling
febf344d33 Remove unnecessary libraries 2020-03-19 18:16:48 +00:00
Alex Ling
ae15398b6c Name the builder stage 2020-03-19 18:14:02 +00:00
Alex Ling
b28f6046dd Merge pull request #17 from WROIATE/master
Update Dockerfile to reduce the image size
2020-03-19 12:29:19 -04:00
Jarao
91b823450c Update Dockerfile 2020-03-19 13:00:11 +08:00
Alex Ling
085fba611c Update README.md 2020-03-17 11:59:32 -04:00
Alex Ling
f8d633c751 Add example library structure to README 2020-03-17 11:45:46 -04:00
Alex Ling
f5e6f42fc2 Update README.md 2020-03-15 13:16:19 -04:00
Alex Ling
3ca6d3d338 Bump version (0.2.0 -> 0.2.1) 2020-03-15 17:09:27 +00:00
Alex Ling
750a28eccb Break words in modal title and path to handle long text 2020-03-15 02:58:27 +00:00
Alex Ling
88b16445e2 Show entry title instead of book title in modal 2020-03-15 02:55:35 +00:00
Alex Ling
7774efa471 When a title has no entry as immediate child, always return 0 as the reading progress 2020-03-15 02:30:18 +00:00
Alex Ling
4aeda53806 Sort title_ids and entries alphanumerically 2020-03-15 02:29:45 +00:00
Alex Ling
5d62a87720 Fix inaccurate sorting when sorting by progress 2020-03-15 02:28:21 +00:00
Alex Ling
e902e1dff0 Merge branch 'nested' into v0.2.1 2020-03-15 02:15:55 +00:00
Alex Ling
9fe32b5011 When a title contains no entry as immediate child, display mango logo and remove progress badge 2020-03-15 02:10:22 +00:00
Alex Ling
e65d701e0a Show sum of entries and titles count when displaying the number of entries 2020-03-15 02:08:20 +00:00
Alex Ling
5a500364fc Show a list of parent directories on the title page 2020-03-15 01:45:10 +00:00
Alex Ling
3e42266955 List the parent title objects in Title.to_json 2020-03-15 01:31:14 +00:00
Alex Ling
6407cea7bf Refactor src/library.cr to reduce memory usage
- Store the `Title` objects in `Library@title_hash`
- The `Title` objects only stores IDs to other titles
2020-03-15 01:05:37 +00:00
Alex Ling
7e22cc5f57 Fix bug in API /api/book/:tid that causes 500 2020-03-15 01:03:49 +00:00
Alex Ling
e68678f2fb Remove unnecessary JSON::Field calls 2020-03-14 23:59:46 +00:00
Alex Ling
82fb45b242 Use json builder in src/library.cr instead of json mapping 2020-03-14 23:58:49 +00:00
Alex Ling
46dfc2f712 Set login cookie expiration date 2020-03-14 22:53:52 +00:00
Alex Ling
79aa816ca8 Merge branch 'v0.2.0' of https://github.com/hkalexling/Mango into v0.2.0 2020-03-13 18:16:14 +00:00
Alex Ling
e35cf2ce0c Update README.md 2020-03-13 14:00:22 -04:00
Alex Ling
47ba0e39af Add dark mode screenshot 2020-03-13 13:59:25 -04:00
Alex Ling
aedb13ac92 Update README.md 2020-03-13 13:53:54 -04:00
Alex Ling
d1c0e52f90 Fix crash after generating default config 2020-03-13 17:46:28 +00:00
Alex Ling
173ff2d2e6 Ignore key mangadex_default in config YAML and remove unnecessary
calls to `YAML::Field`
2020-03-13 17:45:29 +00:00
Alex Ling
ae281e2e21 Bump version number (0.1.2 -> 0.2.0) 2020-03-13 17:03:20 +00:00
Alex Ling
2c10623731 Formatting 2020-03-13 17:03:06 +00:00
Alex Ling
31da5acdc5 Preserve line-breaks in download error messages 2020-03-13 17:00:52 +00:00
Alex Ling
77237a274a Color the close button in alert black so it won't disappear in dark
mode
2020-03-13 01:38:16 +00:00
Alex Ling
318501bc9b Show reading progress in reader 2020-03-13 01:05:16 +00:00
Alex Ling
dc5284968d Use helper function compare_alphanumerically(String, String) to make
the function call shorter
2020-03-12 23:53:51 +00:00
Alex Ling
01216d806c Add a helper function to combine the two steps of filenames sorting
(splitting and comparing)
2020-03-12 23:52:49 +00:00
Alex Ling
c4ffb5cd59 Handle the case when two split arrays have different size in
`compare_alphanumerically`
2020-03-12 23:51:09 +00:00
Alex Ling
50ce0e2b54 Fix typo 2020-03-12 23:49:40 +00:00
Alex Ling
8b8967de26 Simplify the split_by_alphanumeric function 2020-03-12 23:49:02 +00:00
Alex Ling
335fb45de6 Add spec for util.cr 2020-03-12 23:47:14 +00:00
Alex Ling
8c7ced87f1 Add nested library support (WIP) 2020-03-12 20:37:03 +00:00
Alex Ling
00d2540b95 Merge pull request #14 from Leeingnyo/change-page-sort
Sort page names alphanumerically
2020-03-12 13:37:59 -04:00
Leeingnyo
d120433525 Sort page alphanumerically
See https://ux.stackexchange.com/questions/95431/how-should-sorting-work-when-numeric-is-mixed-with-alpha-numeric
2020-03-13 02:27:59 +09:00
Alex Ling
9536ce62e6 Add the "auto" sorting option (#9) 2020-03-12 03:10:56 +00:00
Alex Ling
4ba81b9ffe Sort download jobs listed on the download manager by time 2020-03-11 19:28:04 +00:00
Alex Ling
c355c67415 Download older chapters (as shown on MangaDex) first 2020-03-11 19:26:23 +00:00
Alex Ling
4def23a5cf Fix the problem that URLs are not being parsed on the download page 2020-03-11 18:15:43 +00:00
Alex Ling
943076ccf7 Simplify the download queue tasks APIs 2020-03-11 18:11:32 +00:00
Alex Ling
36034042f2 Fix styling issues with dark theme on the download page 2020-03-11 03:07:09 +00:00
Alex Ling
36e2b2bfaf Add dark mode support 2020-03-07 02:51:08 +00:00
Alex Ling
c6c908953b Use transparent icon 2020-03-07 02:50:44 +00:00
Alex Ling
3ae0ad6348 Add fontawesome and add the "adjust" icon to navigation bar to prepare for dark
mode.
2020-03-06 02:25:10 +00:00
Alex Ling
7ca40215b6 Remove temp DB files before resetting the state 2020-03-04 01:56:48 +00:00
Alex Ling
54206bc6ac Finish the test cases for MangaDex::Queue 2020-03-04 01:52:17 +00:00
Alex Ling
1abdac2fdd Better state management in spec and add some tests for MangaDex::Queue 2020-03-03 18:34:39 +00:00
Alex Ling
9ffe896705 Only log the "baking dist/" message when building for release 2020-03-03 02:51:45 +00:00
Alex Ling
7a7c855ce4 Remove the unused gzip import from static_handler.cr 2020-03-03 02:51:09 +00:00
Alex Ling
e2d01f7eb9 Remove error handling in the parse_query_result method of
MangaDex::Job, and pass the possible exceptions to the frontend and
handle them there.
2020-03-03 02:33:32 +00:00
Alex Ling
7575785c1c Remove the unused get method from MangaDex::Queue 2020-03-03 02:32:08 +00:00
Alex Ling
dfd53bc51d Fix incorrect variable in MangaDex::Downloader (@stop -> @stopped) 2020-03-03 02:06:27 +00:00
Alex Ling
f140ffa4b2 Add and use MLogger in MangaDex::Queue and MangaDex::Downloader 2020-03-03 02:05:55 +00:00
Alex Ling
589483cd75 Center the message on message.ecr 2020-03-02 16:55:08 +00:00
Alex Ling
306edc3c77 Handle HTTP 404 and 500 errors 2020-03-02 16:54:29 +00:00
Alex Ling
30af64e9ca Add pause/resume download button to the download manager 2020-03-02 16:30:05 +00:00
Alex Ling
fecb96c91b Redirect to the download manager after adding jobs to the queue 2020-03-02 02:10:01 +00:00
Alex Ling
4f01aba3e1 Add link to the download manager page on the admin page 2020-03-02 01:57:09 +00:00
Alex Ling
f13f7989d5 Finish the download manager page 2020-03-02 01:50:04 +00:00
Alex Ling
1ce553f541 Add the /admin/downloads page for monitoring download queue 2020-03-01 03:05:40 +00:00
Alex Ling
c4253db572 Add status message to MangaDex::Job and improve formatting 2020-03-01 03:04:55 +00:00
Alex Ling
db6d33eae1 Add sourcerer.io contributors 2020-02-29 21:47:59 -05:00
Alex Ling
8fbc5528a8 Move the JS alert function definition to /js/alert.js and move the alert div to views/layout.ecr 2020-03-01 01:23:16 +00:00
Alex Ling
d50804830d Merge branch 'master' into v0.2.0 2020-02-29 17:50:38 +00:00
Alex Ling
5d7bbc7c9b Remove @log from MangaDex::Job and save the @success_count and
@fail_count instead
2020-02-29 04:11:48 +00:00
Alex Ling
0b463539c9 Better log in MangaDex::API when status is not OK, and handles external
chapters
2020-02-29 04:10:09 +00:00
Alex Ling
7f0088f45a Fix typo (embeded -> embedded) 2020-02-29 00:08:27 +00:00
Alex Ling
5645f272df When an invalid zip file is found, instead of ignoring it silently, log
a warning message
2020-02-28 18:08:56 +00:00
Alex Ling
dc3bbd10d6 Close zip file after listing entries to prevent leaking 2020-02-28 17:55:45 +00:00
Alex Ling
c89c74c71b Bump version to 0.1.2 2020-02-28 17:53:35 +00:00
Alex Ling
cb76a96126 Removing the logging of the library json on scan 2020-02-28 17:52:50 +00:00
Alex Ling
73b38492ba Remove console in minified JS files 2020-02-28 17:52:14 +00:00
Alex Ling
bf37c4aa10 Sorting in library and title page 2020-02-28 17:51:26 +00:00
Alex Ling
f837be0718 Ignore invalid zip files in library 2020-02-28 17:45:30 +00:00
Alex Ling
8c47d50291 Update download API to use Queue in Context and use @full_title 2020-02-28 17:38:45 +00:00
Alex Ling
4ca8daca29 Add API to fetch download queue 2020-02-28 17:38:08 +00:00
Alex Ling
d3d8dff6d2 Add MangaDex::Queue to Context 2020-02-28 17:37:08 +00:00
Alex Ling
f11a5cd608 Finish downloader 2020-02-28 17:36:21 +00:00
Alex Ling
6bccba16da Add MangaDex::PageJob and MangaDex::Downloader 2020-02-28 17:31:09 +00:00
Alex Ling
28ac5c7a00 Formatting in mangadex/api.cr 2020-02-28 17:17:57 +00:00
Alex Ling
f8e0c6d795 Remove download code from mangadex/api.cr, as the download functionality
is now handled by mangadex/downloader.cr
2020-02-28 17:15:55 +00:00
Alex Ling
e3d505d62b Add @full_title to MangaDex::Chapter 2020-02-28 17:14:38 +00:00
Alex Ling
77864afa67 Removing the logging of the library json on scan 2020-02-28 16:55:39 +00:00
Alex Ling
5abdca24c2 Ignore invalid zip files in library 2020-02-28 16:54:37 +00:00
Alex Ling
e8c365b7a1 Rename module Mangadex to MangaDex 2020-02-28 16:48:48 +00:00
Alex Ling
6659041631 Allow missing keys in the "mangadex" dictionary in the config file 2020-02-28 16:24:46 +00:00
Alex Ling
fa50f4cb88 Delete the single job push method 2020-02-26 23:55:38 +00:00
Alex Ling
c39a1ddbaf Merge branch 'master' into v0.2.0 2020-02-26 18:26:49 +00:00
Alex Ling
7de01991a0 Fix the problem that all chapters (regardless of selections) will be
posted to the API
2020-02-26 18:20:14 +00:00
Alex Ling
319967438b Merge branch 'master' into v0.2.0 2020-02-26 18:19:30 +00:00
Alex Ling
1bbb08eede Show spinner and hide download button when posting to API 2020-02-26 17:46:28 +00:00
Alex Ling
d9d1dbc26f Post selected chapter JSON object to API, instead of just posting the
IDs
2020-02-26 17:45:37 +00:00
Alex Ling
c33884ea29 Add mangadex download enpoint to API 2020-02-26 17:42:40 +00:00
Alex Ling
2dd980b92c Add mangadex/downloader.cr (implements MangaDex::Job and
MangaDex::Queue)
2020-02-26 17:41:50 +00:00
Alex Ling
89e747d3ee Get Chapter object directly from chapter ID 2020-02-26 17:31:53 +00:00
Alex Ling
468f109776 Add Manga object as a property of MangaDex::Chapter 2020-02-26 17:31:04 +00:00
Alex Ling
905d02e911 Add queue db path to config.cr 2020-02-26 17:29:34 +00:00
Alex Ling
bb00c2e77f Close zip file after listing entries to prevent leaking 2020-02-26 17:24:16 +00:00
Alex Ling
bc75f4d336 Add unit test 2020-02-25 19:00:10 +00:00
Alex Ling
98baf63b0c Add gitter to README 2020-02-24 22:19:31 -05:00
Alex Ling
46b36860d1 Add static build target to Makefile 2020-02-25 02:32:32 +00:00
Alex Ling
9f6261e02d Make the selection and download buttons on the download page functional 2020-02-25 02:29:00 +00:00
Alex Ling
d782995bac - Filter MangaDex chapters with volume and chapter range
- Select from table rows using jQuery selectable
2020-02-25 01:52:04 +00:00
Alex Ling
b264f7dd76 Remove console in minified JS files 2020-02-24 15:45:05 +00:00
Alex Ling
59622930c7 Fix the problem that minified assets in dist/ are not used. 2020-02-24 15:41:18 +00:00
Alex Ling
e90b97ca43 Display timestamp as time from now using moments.js 2020-02-24 03:14:05 +00:00
Alex Ling
b58d2e3620 Show chapter count 2020-02-24 03:13:47 +00:00
Alex Ling
a507e3be7a Filter in MangaDex search results 2020-02-24 02:56:50 +00:00
Alex Ling
bf0f5270f0 Merge branch 'v0.1.1' 2020-02-23 21:16:23 +00:00
Alex Ling
ac620e1f2a Update docker-compose example 2020-02-23 21:09:50 +00:00
Alex Ling
a7519a791e Fix the problem that minified assets in dist/ are not used. 2020-02-23 19:18:30 +00:00
Alex Ling
7a21f4dc9b Sort titles in library by name by default 2020-02-23 18:36:10 +00:00
Alex Ling
650ebc7f9d Fix #6 2020-02-23 18:35:27 +00:00
Alex Ling
5b34c05243 Use Babel, so I can write modern JS and save my sanity 2020-02-23 18:26:57 +00:00
Alex Ling
803fc8c44b Split routes in server.cr into small files 2020-02-23 18:24:32 +00:00
Alex Ling
67d3d2bd55 Sorting in library and title page 2020-02-22 18:16:04 +00:00
Alex Ling
5ec35f3af6 Sort titles in library by name by default 2020-02-22 18:15:40 +00:00
Alex Ling
dd49f75079 Update README.md 2020-02-21 21:22:55 -05:00
Alex Ling
6be9c3eac6 Update README.md 2020-02-21 21:21:34 -05:00
Alex Ling
0fa95959a7 Basic search functionality 2020-02-22 02:14:01 +00:00
Alex Ling
83597e7f84 Fix typo 2020-02-22 02:13:35 +00:00
Alex Ling
c893135ec6 Use Babel, so I can write mordern JS and save my sanity 2020-02-22 02:12:01 +00:00
Alex Ling
5a2f80b5e1 Add issue templates 2020-02-21 16:42:12 -05:00
Alex Ling
5b4d79220c Provide pre-built binary (amd64) in README 2020-02-21 14:08:55 -05:00
Alex Ling
a3356344fa Add API for fetching manga info from ID 2020-02-21 02:00:01 +00:00
Alex Ling
aecac748dc Add /download page 2020-02-21 01:59:37 +00:00
Alex Ling
c449a1e9b1 Add /download page 2020-02-21 01:59:09 +00:00
Alex Ling
f9a4698fca Add mangadex to config 2020-02-21 01:58:45 +00:00
Alex Ling
676f2ae032 Include chapter information in MangaDex::Manga#to_info_json output 2020-02-21 01:56:57 +00:00
Alex Ling
fd342fe1ee Add mangadex lang code csv to src/assets 2020-02-20 23:41:40 +00:00
Alex Ling
1649f286aa Split routes in server.cr into small files 2020-02-20 23:39:54 +00:00
Alex Ling
60a1032f71 add a basic Mangadex API wrapper 2020-02-20 01:32:18 +00:00
61 changed files with 2721 additions and 448 deletions

3
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,3 @@
# These are supported funding model platforms
patreon: hkalexling

32
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,32 @@
---
name: Bug report
about: I found a bug in Mango!
title: "[Bug Report]"
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Environment (please complete the following information):**
- OS: [e.g. Ubuntu 18.04]
- Browser [e.g. chrome, safari, if applicable]
- Mango Version [e.g. v0.1.0]
**Docker (if you are running Mango in a Docker container)**
- The `docker-compose.yml` file you are using
**Additional context**
Add any other context about the problem here. Add screenshots if applicable.

View File

@@ -0,0 +1,17 @@
---
name: Feature request
about: Suggest a feature for Mango
title: "[Feature Request]"
labels: enhancement
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Additional context**
Add any other context or screenshots about the feature request here.

View File

@@ -0,0 +1,10 @@
---
name: General Question
about: I have a question about Mango
title: "[Question]"
labels: general question
assignees: ''
---

BIN
.github/screenshots/dark.png vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 598 KiB

24
.github/workflows/build.yml vendored Normal file
View File

@@ -0,0 +1,24 @@
name: Build
on:
push:
branches: [ master, dev ]
pull_request:
branches: [ master, dev ]
jobs:
build:
runs-on: ubuntu-latest
container:
image: crystallang/crystal:0.32.1-alpine
steps:
- uses: actions/checkout@v2
- name: Install dependencies
run: apk add --no-cache yarn yaml sqlite-static
- name: Build
run: make
- name: Run tests
run: make test

View File

@@ -1,18 +1,16 @@
FROM crystallang/crystal:0.32.0
RUN apt-get update && apt-get install -y curl
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash -
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y nodejs yarn libsqlite3-dev
FROM crystallang/crystal:0.32.1-alpine AS builder
WORKDIR /Mango
COPY . .
COPY package*.json .
RUN apk add --no-cache yarn yaml sqlite-static \
&& make static
RUN make && make install
FROM library/alpine
CMD ["mango"]
WORKDIR /
COPY --from=builder /Mango/mango .
CMD ["./mango"]

View File

@@ -10,12 +10,18 @@ uglify:
build: libs
crystal build src/mango.cr --release --progress
static: uglify | libs
crystal build src/mango.cr --release --progress --static
libs:
shards install
run:
crystal run src/mango.cr --error-trace
test:
crystal spec
install:
cp mango $(INSTALL_DIR)/mango

View File

@@ -1,17 +1,29 @@
# Mango
![banner](./public/img/banner-paddings.png)
# Mango
[![Patreon](https://img.shields.io/badge/support-patreon-brightgreen?link=https://www.patreon.com/hkalexling)](https://www.patreon.com/hkalexling) ![Build](https://github.com/hkalexling/Mango/workflows/Build/badge.svg) [![Gitter](https://badges.gitter.im/mango-cr/mango.svg)](https://gitter.im/mango-cr/mango?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
Mango is a self-hosted manga server and reader. Its features include
- Multi-user support
- Dark/light mode switch
- Supports both `.zip` and `.cbz` formats
- Supports nested folders in library
- Automatically stores reading progress
- Built-in [MangaDex](https://mangadex.org/) downloader
- The web reader is responsive and works well on mobile, so there is no need for a mobile app
- All the static files are embedded in the binary, so the deployment process is easy and painless
## Installation
### Pre-built Binary
1. Simply download the pre-built binary file `mango` for the latest [release](https://github.com/hkalexling/Mango/releases). All the dependencies are statically linked, and it should work with most Linux systems on amd64.
### Docker
1. Make sure you have docker installed and running. You will also need `docker-compose`
@@ -21,10 +33,9 @@ Mango is a self-hosted manga server and reader. Its features include
5. Run `docker-compose up`. This should build the docker image and start the container with Mango running inside
6. Head over to `localhost:9000` to log in
### Build from source
1. Make sure you have Crystal, Node and Yarn installed
1. Make sure you have Crystal, Node and Yarn installed. You might also need to install the development headers for `libsqlite3` and `libyaml`.
2. Clone the repository
3. `make && sudo make install`
4. Start Mango by running the command `mango`
@@ -35,7 +46,7 @@ Mango is a self-hosted manga server and reader. Its features include
### CLI
```
Mango e-manga server/reader. Version 0.1.0
Mango e-manga server/reader. Version 0.2.0
-v, --version Show version
-h, --help Show help
@@ -53,25 +64,34 @@ library_path: ~/mango/library
db_path: ~/mango/mango.db
scan_interval_minutes: 5
log_level: info
mangadex:
base_url: https://mangadex.org
api_url: https://mangadex.org/api
download_wait_seconds: 5
download_retries: 4
download_queue_db_path: ~/mango/queue.db
```
- `scan_interval_minutes` can be any non-negative integer. Setting it to `0` disables the periodic scan
- `log_level` can be `debug`, `info`, `warn`, `error`, `fatal` or `off`. Setting it to `off` disables the logging
### Required Library Structure
### Library Structure
Please make sure that your library directory has the following structure:
You can organize your `.cbz/.zip` files in nested folders in the library directory. Here's an example:
```
.
├── Manga 1
│   └── Manga 1.cbz
│   ├── Volume 1.cbz
│   ├── Volume 2.cbz
│   ├── Volume 3.cbz
│   └── Volume 4.zip
└── Manga 2
├── Vol 0001.zip
├── Vol 0002.zip
├── Vol 0003.zip
├── Vol 0004.zip
└── Vol 0005.zip
   └── Vol. 1
   └── Ch.1 - Ch.3
   ├── 1.zip
   ├── 2.zip
   └── 3.zip
```
### Initial Login
@@ -88,6 +108,10 @@ Title:
![title screenshot](./.github/screenshots/title.png)
Dark mode:
![dark mode screeshot](./.github/screenshots/dark.png)
Reader:
![reader screenshot](./.github/screenshots/reader.png)
@@ -95,3 +119,7 @@ Reader:
Mobile UI:
![mobile screenshot](./.github/screenshots/mobile.png)
## Contributors
[![](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/images/0)](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/links/0)[![](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/images/1)](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/links/1)[![](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/images/2)](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/links/2)[![](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/images/3)](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/links/3)[![](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/images/4)](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/links/4)[![](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/images/5)](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/links/5)[![](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/images/6)](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/links/6)[![](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/images/7)](https://sourcerer.io/fame/hkalexling/hkalexling/Mango/links/7)

View File

@@ -11,5 +11,5 @@ services:
ports:
- 9000:9000
volumes:
- ./mango:/root/mango
- ./config:/root/.config/mango
- ~/mango:/root/mango
- ~/.config/mango:/root/.config/mango

View File

@@ -1,10 +1,12 @@
const gulp = require('gulp');
const uglify = require('gulp-uglify');
const minify = require("gulp-babel-minify");
const minifyCss = require('gulp-minify-css');
gulp.task('minify-js', () => {
return gulp.src('public/js/*.js')
.pipe(uglify())
.pipe(minify({
removeConsole: true
}))
.pipe(gulp.dest('dist/js'));
});

View File

@@ -7,8 +7,8 @@
"license": "MIT",
"devDependencies": {
"gulp": "^4.0.2",
"gulp-minify-css": "^1.2.4",
"gulp-uglify": "^3.0.2"
"gulp-babel-minify": "^0.5.1",
"gulp-minify-css": "^1.2.4"
},
"scripts": {
"uglify": "gulp"

View File

@@ -1,9 +1,24 @@
.uk-alert-close {
color: black !important;
}
.uk-card-body {
padding: 20px;
}
.uk-card-media-top {
max-height: 350px;
overflow: hidden;
height: 250px;
}
@media (min-width: 600px) {
.uk-card-media-top {
height: 300px;
}
}
.uk-card-media-top > img {
height: 100%;
width: 100%;
object-fit: cover;
}
.uk-card-title {
height: 3em;
}
.acard:hover {
text-decoration: none;
@@ -17,7 +32,7 @@
#scan-status {
cursor: auto;
}
.uk-card-title {
.break-word {
word-wrap: break-word;
}
.uk-logo > img {
@@ -26,3 +41,18 @@
.uk-search {
width: 100%;
}
#selectable .ui-selecting {
background: #EEE6B9;
}
#selectable .ui-selected {
background: #F4E487;
}
#selectable .ui-selecting.dark {
background: #5E5731;
}
#selectable .ui-selected.dark {
background: #9D9252;
}
td > .uk-dropdown {
white-space: pre-line;
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

6
public/js/alert.js Normal file
View File

@@ -0,0 +1,6 @@
const alert = (level, text) => {
$('#alert').empty();
const html = `<div class="uk-alert-${level}" uk-alert><a class="uk-alert-close" uk-close></a><p>${text}</p></div>`;
$('#alert').append(html);
$("html, body").animate({ scrollTop: 0 });
};

18
public/js/dots.js Normal file
View File

@@ -0,0 +1,18 @@
const truncate = () => {
$('.acard .uk-card-title').each((i, e) => {
$(e).dotdotdot({
truncate: 'letter',
watch: true,
callback: (truncated) => {
if (truncated) {
$(e).attr('uk-tooltip', $(e).attr('data-title'));
}
else {
$(e).removeAttr('uk-tooltip');
}
}
});
});
};
truncate();

View File

@@ -0,0 +1,138 @@
$(() => {
$('input.uk-checkbox').each((i, e) => {
$(e).change(() => {
loadConfig();
});
});
loadConfig();
load();
const intervalMS = 5000;
setTimeout(() => {
setInterval(() => {
if (globalConfig.autoRefresh !== true) return;
load();
}, intervalMS);
}, intervalMS);
});
var globalConfig = {};
var loading = false;
const loadConfig = () => {
globalConfig.autoRefresh = $('#auto-refresh').prop('checked');
};
const remove = (id) => {
var url = '/api/admin/mangadex/queue/delete';
if (id !== undefined)
url += '?' + $.param({id: id});
console.log(url);
$.ajax({
type: 'POST',
url: url,
dataType: 'json'
})
.done(data => {
if (!data.success && data.error) {
alert('danger', `Failed to remove job from download queue. Error: ${data.error}`);
return;
}
load();
})
.fail((jqXHR, status) => {
alert('danger', `Failed to remove job from download queue. Error: [${jqXHR.status}] ${jqXHR.statusText}`);
});
};
const refresh = (id) => {
var url = '/api/admin/mangadex/queue/retry';
if (id !== undefined)
url += '?' + $.param({id: id});
console.log(url);
$.ajax({
type: 'POST',
url: url,
dataType: 'json'
})
.done(data => {
if (!data.success && data.error) {
alert('danger', `Failed to restart download job. Error: ${data.error}`);
return;
}
load();
})
.fail((jqXHR, status) => {
alert('danger', `Failed to restart download job. Error: [${jqXHR.status}] ${jqXHR.statusText}`);
});
};
const toggle = () => {
$('#pause-resume-btn').attr('disabled', '');
const paused = $('#pause-resume-btn').text() === 'Resume download';
const action = paused ? 'resume' : 'pause';
const url = `/api/admin/mangadex/queue/${action}`;
$.ajax({
type: 'POST',
url: url,
dataType: 'json'
})
.fail((jqXHR, status) => {
alert('danger', `Failed to ${action} download queue. Error: [${jqXHR.status}] ${jqXHR.statusText}`);
})
.always(() => {
load();
$('#pause-resume-btn').removeAttr('disabled');
});
};
const load = () => {
if (loading) return;
loading = true;
console.log('fetching');
$.ajax({
type: 'GET',
url: '/api/admin/mangadex/queue',
dataType: 'json'
})
.done(data => {
if (!data.success && data.error) {
alert('danger', `Failed to fetch download queue. Error: ${data.error}`);
return;
}
console.log(data);
const btnText = data.paused ? "Resume download" : "Pause download";
$('#pause-resume-btn').text(btnText);
$('#pause-resume-btn').removeAttr('hidden');
const rows = data.jobs.map(obj => {
var cls = 'uk-label ';
if (obj.status === 'Completed')
cls += 'uk-label-success';
if (obj.status === 'Error')
cls += 'uk-label-danger';
if (obj.status === 'MissingPages')
cls += 'uk-label-warning';
const info = obj.status_message.length > 0 ? '<span uk-icon="info"></span>' : '';
const statusSpan = `<span class="${cls}">${obj.status} ${info}</span>`;
const dropdown = obj.status_message.length > 0 ? `<div uk-dropdown>${obj.status_message}</div>` : '';
const retryBtn = obj.status_message.length > 0 ? `<a onclick="refresh('${obj.id}')" uk-icon="refresh"></a>` : '';
return `<tr id="chapter-${obj.id}">
<td><a href="${baseURL}/chapter/${obj.id}">${obj.title}</a></td>
<td><a href="${baseURL}/manga/${obj.manga_id}">${obj.manga_title}</a></td>
<td>${obj.success_count}/${obj.pages}</td>
<td>${moment(obj.time).fromNow()}</td>
<td>${statusSpan} ${dropdown}</td>
<td>
<a onclick="remove('${obj.id}')" uk-icon="trash"></a>
${retryBtn}
</td>
</tr>`;
});
const tbody = `<tbody>${rows.join('')}</tbody>`;
$('tbody').remove();
$('table').append(tbody);
})
.fail((jqXHR, status) => {
alert('danger', `Failed to fetch download queue. Error: [${jqXHR.status}] ${jqXHR.statusText}`);
})
.always(() => {
loading = false;
});
};

299
public/js/download.js Normal file
View File

@@ -0,0 +1,299 @@
$(() => {
$('#search-input').keypress(event => {
if (event.which === 13) {
search();
}
});
$('.filter-field').each((i, ele) => {
$(ele).change(() => {
buildTable();
});
});
});
const selectAll = () => {
$('tbody > tr').each((i, e) => {
$(e).addClass('ui-selected');
});
};
const unselect = () => {
$('tbody > tr').each((i, e) => {
$(e).removeClass('ui-selected');
});
};
const download = () => {
const selected = $('tbody > tr.ui-selected');
if (selected.length === 0) return;
UIkit.modal.confirm(`Download ${selected.length} selected chapters?`).then(() => {
$('#download-btn').attr('hidden', '');
$('#download-spinner').removeAttr('hidden');
const ids = selected.map((i, e) => {
return $(e).find('td').first().text();
}).get();
const chapters = globalChapters.filter(c => ids.indexOf(c.id) >= 0);
console.log(ids);
$.ajax({
type: 'POST',
url: '/api/admin/mangadex/download',
data: JSON.stringify({chapters: chapters}),
contentType: "application/json",
dataType: 'json'
})
.done(data => {
console.log(data);
if (data.error) {
alert('danger', `Failed to add chapters to the download queue. Error: ${data.error}`);
return;
}
const successCount = parseInt(data.success);
const failCount = parseInt(data.fail);
UIkit.modal.confirm(`${successCount} of ${successCount + failCount} chapters added to the download queue. Proceed to the download manager?`).then(() => {
window.location.href = '/admin/downloads';
});
styleModal();
})
.fail((jqXHR, status) => {
alert('danger', `Failed to add chapters to the download queue. Error: [${jqXHR.status}] ${jqXHR.statusText}`);
})
.always(() => {
$('#download-spinner').attr('hidden', '');
$('#download-btn').removeAttr('hidden');
});
});
styleModal();
};
const toggleSpinner = () => {
var attr = $('#spinner').attr('hidden');
if (attr) {
$('#spinner').removeAttr('hidden');
$('#search-btn').attr('hidden', '');
}
else {
$('#search-btn').removeAttr('hidden');
$('#spinner').attr('hidden', '');
}
searching = !searching;
};
var searching = false;
var globalChapters;
const search = () => {
if (searching) {
return;
}
$('#manga-details').attr('hidden', '');
$('#filter-form').attr('hidden', '');
$('table').attr('hidden', '');
$('#selection-controls').attr('hidden', '');
$('#filter-notification').attr('hidden', '');
toggleSpinner();
const input = $('input').val();
if (input === "") {
toggleSpinner();
return;
}
var int_id = -1;
try {
const path = new URL(input).pathname;
const match = /\/title\/([0-9]+)/.exec(path);
int_id = parseInt(match[1]);
}
catch(e) {
int_id = parseInt(input);
}
if (int_id <= 0 || isNaN(int_id)) {
alert('danger', 'Please make sure you are using a valid manga ID or manga URL from Mangadex.');
toggleSpinner();
return;
}
$.getJSON("/api/admin/mangadex/manga/" + int_id)
.done((data) => {
if (data.error) {
alert('danger', 'Failed to get manga info. Error: ' + data.error);
return;
}
const cover = baseURL + data.cover_url;
$('#cover').attr("src", cover);
$('#title').text("Title: " + data.title);
$('#artist').text("Artist: " + data.artist);
$('#author').text("Author: " + data.author);
$('#manga-details').removeAttr('hidden');
console.log(data.chapters);
globalChapters = data.chapters;
let langs = new Set();
let group_names = new Set();
data.chapters.forEach(chp => {
Object.entries(chp.groups).forEach(([k, v]) => {
group_names.add(k);
});
langs.add(chp.language);
});
const comp = (a, b) => {
var ai;
var bi;
try {ai = parseFloat(a);} catch(e) {}
try {bi = parseFloat(b);} catch(e) {}
if (typeof ai === 'undefined') return -1;
if (typeof bi === 'undefined') return 1;
if (ai < bi) return 1;
if (ai > bi) return -1;
return 0;
};
langs = [...langs].sort();
group_names = [...group_names].sort();
langs.unshift('All');
group_names.unshift('All');
$('select#lang-select').append(langs.map(e => `<option>${e}</option>`).join(''));
$('select#group-select').append(group_names.map(e => `<option>${e}</option>`).join(''));
$('#filter-form').removeAttr('hidden');
buildTable();
})
.fail((jqXHR, status) => {
alert('danger', `Failed to get manga info. Error: [${jqXHR.status}] ${jqXHR.statusText}`);
})
.always(() => {
toggleSpinner();
});
};
const parseRange = str => {
const regex = /^[\t ]*(?:(?:(<|<=|>|>=)[\t ]*([0-9]+))|(?:([0-9]+))|(?:([0-9]+)[\t ]*-[\t ]*([0-9]+))|(?:[\t ]*))[\t ]*$/m;
const matches = str.match(regex);
var num;
if (!matches) {
alert('danger', `Failed to parse filter input ${str}`);
return [null, null];
}
else if (typeof matches[1] !== 'undefined' && typeof matches[2] !== 'undefined') {
// e.g., <= 30
num = parseInt(matches[2]);
if (isNaN(num)) {
alert('danger', `Failed to parse filter input ${str}`);
return [null, null];
}
switch (matches[1]) {
case '<':
return [null, num - 1];
case '<=':
return [null, num];
case '>':
return [num + 1, null];
case '>=':
return [num, null];
}
}
else if (typeof matches[3] !== 'undefined') {
// a single number
num = parseInt(matches[3]);
if (isNaN(num)) {
alert('danger', `Failed to parse filter input ${str}`);
return [null, null];
}
return [num, num];
}
else if (typeof matches[4] !== 'undefined' && typeof matches[5] !== 'undefined') {
// e.g., 10 - 23
num = parseInt(matches[4]);
const n2 = parseInt(matches[5]);
if (isNaN(num) || isNaN(n2) || num > n2) {
alert('danger', `Failed to parse filter input ${str}`);
return [null, null];
}
return [num, n2];
}
else {
// empty or space only
return [null, null];
}
};
const getFilters = () => {
const filters = {};
$('.uk-select').each((i, ele) => {
const id = $(ele).attr('id');
const by = id.split('-')[0];
const choice = $(ele).val();
filters[by] = choice;
});
filters.volume = parseRange($('#volume-range').val());
filters.chapter = parseRange($('#chapter-range').val());
return filters;
};
const buildTable = () => {
$('table').attr('hidden', '');
$('#selection-controls').attr('hidden', '');
$('#filter-notification').attr('hidden', '');
console.log('rebuilding table');
const filters = getFilters();
console.log('filters:', filters);
var chapters = globalChapters.slice();
Object.entries(filters).forEach(([k, v]) => {
if (v === 'All') return;
if (k === 'group') {
chapters = chapters.filter(c => v in c.groups);
return;
}
if (k === 'lang') {
chapters = chapters.filter(c => c.language === v);
return;
}
const lb = parseFloat(v[0]);
const ub = parseFloat(v[1]);
if (isNaN(lb) && isNaN(ub)) return;
chapters = chapters.filter(c => {
const val = parseFloat(c[k]);
if (isNaN(val)) return false;
if (isNaN(lb))
return val <= ub;
else if (isNaN(ub))
return val >= lb;
else
return val >= lb && val <= ub;
});
});
console.log('filtered chapters:', chapters);
$('#count-text').text(`${chapters.length} chapters found`);
const chaptersLimit = 1000;
if (chapters.length > chaptersLimit) {
$('#filter-notification').text(`Mango can only list ${chaptersLimit} chapters, but we found ${chapters.length} chapters in this manga. Please use the filter options above to narrow down your search.`);
$('#filter-notification').removeAttr('hidden');
return;
}
const inner = chapters.map(chp => {
const group_str = Object.entries(chp.groups).map(([k, v]) => {
return `<a href="${baseURL }/group/${v}">${k}</a>`;
}).join(' | ');
const dark = getTheme() === 'dark' ? 'dark' : '';
return `<tr class="ui-widget-content ${dark}">
<td><a href="${baseURL}/chapter/${chp.id}">${chp.id}</a></td>
<td>${chp.title}</td>
<td>${chp.language}</td>
<td>${group_str}</td>
<td>${chp.volume}</td>
<td>${chp.chapter}</td>
<td>${moment.unix(chp.time).fromNow()}</td>
</tr>`;
}).join('');
const tbody = `<tbody id="selectable">${inner}</tbody>`;
$('tbody').remove();
$('table').append(tbody);
$('table').removeAttr('hidden');
$("#selectable").selectable({
filter: 'tr'
});
$('#selection-controls').removeAttr('hidden');
};

5
public/js/fontawesome.min.js vendored Normal file

File diff suppressed because one or more lines are too long

View File

@@ -58,8 +58,12 @@ $('#page-select').change(function(){
jumpTo(parseInt($('#page-select').val()));
});
function showControl(idx) {
const pageCount = $('#page-select > option').length;
const progressText = `Progress: ${idx}/${pageCount} (${(idx/pageCount * 100).toFixed(1)}%)`;
$('#progress-label').text(progressText);
$('#page-select').val(idx);
UIkit.modal($('#modal-sections')).show();
styleModal();
}
function jumpTo(page) {
var ary = window.location.pathname.split('/');

5
public/js/solid.min.js vendored Normal file

File diff suppressed because one or more lines are too long

124
public/js/sort-items.js Normal file
View File

@@ -0,0 +1,124 @@
$(() => {
const sortItems = () => {
const sort = $('#sort-select').find(':selected').attr('id');
const ary = sort.split('-');
const by = ary[0];
const dir = ary[1];
let items = $('.item');
items.remove();
const ctxAry = [];
const keyRange = {};
if (by === 'auto') {
// intelligent sorting
items.each((i, item) => {
const name = $(item).find('.uk-card-title').text();
const regex = /([^0-9\n\r\ ]*)[ ]*([0-9]*\.*[0-9]+)/g;
const numbers = {};
let match = regex.exec(name);
while (match) {
const key = match[1];
const num = parseFloat(match[2]);
numbers[key] = num;
if (!keyRange[key]) {
keyRange[key] = [num, num, 1];
}
else {
keyRange[key][2] += 1;
if (num < keyRange[key][0]) {
keyRange[key][0] = num;
}
else if (num > keyRange[key][1]) {
keyRange[key][1] = num;
}
}
match = regex.exec(name);
}
ctxAry.push({index: i, numbers: numbers});
});
console.log(keyRange);
const sortedKeys = Object.keys(keyRange).filter(k => {
return keyRange[k][2] >= items.length / 2;
});
sortedKeys.sort((a, b) => {
// sort by frequency of the key first
if (keyRange[a][2] !== keyRange[b][2]) {
return keyRange[a][2] < keyRange[b][2];
}
// then sort by range of the key
return (keyRange[a][1] - keyRange[a][0]) < (keyRange[b][1] - keyRange[b][0]);
});
console.log(sortedKeys);
ctxAry.sort((a, b) => {
for (let i = 0; i < sortedKeys.length; i++) {
const key = sortedKeys[i];
if (a.numbers[key] === undefined && b.numbers[key] === undefined)
continue;
if (a.numbers[key] === undefined)
return 1;
if (b.numbers[key] === undefined)
return -1;
if (a.numbers[key] === b.numbers[key])
continue;
return a.numbers[key] > b.numbers[key];
}
return 0;
});
const sortedItems = [];
ctxAry.forEach(ctx => {
sortedItems.push(items[ctx.index]);
});
items = sortedItems;
if (dir === 'down') {
items.reverse();
}
}
else {
items.sort((a, b) => {
var res;
if (by === 'name')
res = $(a).find('.uk-card-title').text() > $(b).find('.uk-card-title').text();
else if (by === 'date')
res = $(a).attr('data-mtime') > $(b).attr('data-mtime');
else if (by === 'progress') {
const ap = parseFloat($(a).attr('data-progress'));
const bp = parseFloat($(b).attr('data-progress'));
if (ap === bp)
// if progress is the same, we compare by name
res = $(a).find('.uk-card-title').text() > $(b).find('.uk-card-title').text();
else
res = ap > bp;
}
if (dir === 'up')
return res;
else
return !res;
});
}
var html = '';
$('#item-container').append(items);
};
$('#sort-select').change(() => {
sortItems();
});
if ($('option#auto-up').length > 0)
$('option#auto-up').attr('selected', '');
else
$('option#name-up').attr('selected', '');
sortItems();
});

43
public/js/theme.js Normal file
View File

@@ -0,0 +1,43 @@
const getTheme = () => {
var theme = localStorage.getItem('theme');
if (!theme) theme = 'light';
return theme;
};
const saveTheme = theme => {
localStorage.setItem('theme', theme);
};
const toggleTheme = () => {
const theme = getTheme();
const newTheme = theme === 'dark' ? 'light' : 'dark';
setTheme(newTheme);
saveTheme(newTheme);
};
const setTheme = themeStr => {
if (themeStr === 'dark') {
$('html').css('background', 'rgb(20, 20, 20)');
$('body').addClass('uk-light');
$('.uk-card').addClass('uk-card-secondary');
$('.uk-card').removeClass('uk-card-default');
$('.ui-widget-content').addClass('dark');
}
else {
$('html').css('background', '');
$('body').removeClass('uk-light');
$('.uk-card').removeClass('uk-card-secondary');
$('.uk-card').addClass('uk-card-default');
$('.ui-widget-content').removeClass('dark');
}
};
const styleModal = () => {
const color = getTheme() === 'dark' ? '#222' : '';
$('.uk-modal-header').css('background', color);
$('.uk-modal-body').css('background', color);
$('.uk-modal-footer').css('background', color);
};
// do it before document is ready to prevent the initial flash of white
setTheme(getTheme());

View File

@@ -1,4 +1,7 @@
function showModal(title, zipPath, pages, percentage, title, entry) {
function showModal(encodedPath, pages, percentage, encodedeTitle, encodedEntryTitle, titleID, entryID) {
const zipPath = decodeURIComponent(encodedPath);
const title = decodeURIComponent(encodedeTitle);
const entry = decodeURIComponent(encodedEntryTitle);
$('#modal button, #modal a').each(function(){
$(this).removeAttr('hidden');
});
@@ -12,24 +15,25 @@ function showModal(title, zipPath, pages, percentage, title, entry) {
if (percentage === 100) {
$('#read-btn').attr('hidden', '');
}
$('#modal-title').text(title);
$('#modal-title').text(entry);
$('#path-text').text(zipPath);
$('#pages-text').text(pages + ' pages');
$('#beginning-btn').attr('href', '/reader/' + title + '/' + entry + '/1');
$('#continue-btn').attr('href', '/reader/' + title + '/' + entry);
$('#beginning-btn').attr('href', '/reader/' + titleID + '/' + entryID + '/1');
$('#continue-btn').attr('href', '/reader/' + titleID + '/' + entryID);
$('#read-btn').click(function(){
updateProgress(title, entry, pages);
updateProgress(titleID, entryID, pages);
});
$('#unread-btn').click(function(){
updateProgress(title, entry, 0);
updateProgress(titleID, entryID, 0);
});
UIkit.modal($('#modal')).show();
styleModal();
}
function updateProgress(title, entry, page) {
$.post('/api/progress/' + title + '/' + entry + '/' + page, function(data) {
function updateProgress(titleID, entryID, page) {
$.post('/api/progress/' + titleID + '/' + entryID + '/' + page, function(data) {
if (data.success) {
location.reload();
}
@@ -39,11 +43,3 @@ function updateProgress(title, entry, page) {
}
});
}
function alert(level, text) {
hideAlert();
var html = '<div class="uk-alert-' + level + '" uk-alert><a class="uk-alert-close" uk-close></a><p>' + text + '</p></div>';
$('#alert').append(html);
}
function hideAlert() {
$('#alert').empty();
}

View File

@@ -1,16 +1,6 @@
$(function(){
$(() => {
var target = '/admin/user/edit';
if (username) target += username;
$('form').attr('action', target);
function alert(level, text) {
hideAlert();
var html = '<div class="uk-alert-' + level + '" uk-alert><a class="uk-alert-close" uk-close></a><p>' + text + '</p></div>';
$('#alert').append(html);
}
function hideAlert() {
$('#alert').empty();
}
if (error) alert('danger', error);
});

View File

@@ -1,11 +1,3 @@
function alert(level, text) {
hideAlert();
var html = '<div class="uk-alert-' + level + '" uk-alert><a class="uk-alert-close" uk-close></a><p>' + text + '</p></div>';
$('#alert').append(html);
}
function hideAlert() {
$('#alert').empty();
}
function remove(username) {
$.post('/api/admin/user/delete/' + username, function(data) {
if (data.success) {

View File

@@ -1,5 +1,5 @@
name: mango
version: 0.1.0
version: 0.2.4
authors:
- Alex Ling <hkalexling@gmail.com>

View File

@@ -0,0 +1,2 @@
---
port: 3000

14
spec/config_spec.cr Normal file
View File

@@ -0,0 +1,14 @@
require "./spec_helper"
describe Config do
it "creates config if it does not exist" do
with_default_config do |config, logger, path|
File.exists?(path).should be_true
end
end
it "correctly loads config" do
config = Config.load "spec/asset/test-config.yml"
config.port.should eq 3000
end
end

105
spec/mangadex_spec.cr Normal file
View File

@@ -0,0 +1,105 @@
require "./spec_helper"
include MangaDex
describe Queue do
it "creates DB at given path" do
with_queue do |queue, path|
File.exists?(path).should be_true
end
end
it "pops nil when empty" do
with_queue do |queue|
queue.pop.should be_nil
end
end
it "inserts multiple jobs" do
with_queue do |queue|
j1 = Job.new "1", "1", "title", "manga_title", JobStatus::Error,
Time.utc
j2 = Job.new "2", "2", "title", "manga_title", JobStatus::Completed,
Time.utc
j3 = Job.new "3", "3", "title", "manga_title", JobStatus::Pending,
Time.utc
j4 = Job.new "4", "4", "title", "manga_title",
JobStatus::Downloading, Time.utc
count = queue.push [j1, j2, j3, j4]
count.should eq 4
end
end
it "pops pending job" do
with_queue do |queue|
job = queue.pop
job.should_not be_nil
job.not_nil!.id.should eq "3"
end
end
it "correctly counts jobs" do
with_queue do |queue|
queue.count.should eq 4
end
end
it "deletes job" do
with_queue do |queue|
queue.delete "4"
queue.count.should eq 3
end
end
it "sets status" do
with_queue do |queue|
job = queue.pop.not_nil!
queue.set_status JobStatus::Downloading, job
job = queue.pop
job.should_not be_nil
job.not_nil!.status.should eq JobStatus::Downloading
end
end
it "sets number of pages" do
with_queue do |queue|
job = queue.pop.not_nil!
queue.set_pages 100, job
job = queue.pop
job.should_not be_nil
job.not_nil!.pages.should eq 100
end
end
it "adds fail/success counts" do
with_queue do |queue|
job = queue.pop.not_nil!
queue.add_success job
queue.add_success job
queue.add_fail job
job = queue.pop
job.should_not be_nil
job.not_nil!.success_count.should eq 2
job.not_nil!.fail_count.should eq 1
end
end
it "appends status message" do
with_queue do |queue|
job = queue.pop.not_nil!
queue.add_message "hello", job
queue.add_message "world", job
job = queue.pop
job.should_not be_nil
job.not_nil!.status_message.should eq "\nhello\nworld"
end
end
it "cleans up" do
with_queue do
true
end
State.reset
end
end

65
spec/spec_helper.cr Normal file
View File

@@ -0,0 +1,65 @@
require "spec"
require "../src/context"
require "../src/server"
class State
@@hash = {} of String => String
def self.get(key)
@@hash[key]?
end
def self.get!(key)
@@hash[key]
end
def self.set(key, value)
return if value.nil?
@@hash[key] = value
end
def self.reset
@@hash.clear
end
end
def get_tempfile(name)
path = State.get name
if path.nil? || !File.exists? path
file = File.tempfile name
State.set name, file.path
return file
else
return File.new path
end
end
def with_default_config
temp_config = get_tempfile "mango-test-config"
config = Config.load temp_config.path
logger = MLogger.new config
yield config, logger, temp_config.path
temp_config.delete
end
def with_storage
with_default_config do |config, logger|
temp_db = get_tempfile "mango-test-db"
storage = Storage.new temp_db.path, logger
clear = yield storage, temp_db.path
if clear == true
temp_db.delete
end
end
end
def with_queue
with_default_config do |config, logger|
temp_queue_db = get_tempfile "mango-test-queue-db"
queue = MangaDex::Queue.new temp_queue_db.path, logger
clear = yield queue, temp_queue_db.path
if clear == true
temp_queue_db.delete
end
end
end

91
spec/storage_spec.cr Normal file
View File

@@ -0,0 +1,91 @@
require "./spec_helper"
describe Storage do
it "creates DB at given path" do
with_storage do |storage, path|
File.exists?(path).should be_true
end
end
it "deletes user" do
with_storage do |storage|
storage.delete_user "admin"
end
end
it "creates new user" do
with_storage do |storage|
storage.new_user "user", "123456", false
storage.new_user "admin", "123456", true
end
end
it "verifies username/password combination" do
with_storage do |storage|
user_token = storage.verify_user "user", "123456"
admin_token = storage.verify_user "admin", "123456"
user_token.should_not be_nil
admin_token.should_not be_nil
State.set "user_token", user_token
State.set "admin_token", admin_token
end
end
it "rejects duplicate username" do
with_storage do |storage|
expect_raises SQLite3::Exception,
"UNIQUE constraint failed: users.username" do
storage.new_user "admin", "123456", true
end
end
end
it "verifies token" do
with_storage do |storage|
user_token = State.get! "user_token"
user = storage.verify_token user_token
user.should eq "user"
end
end
it "verfies admin token" do
with_storage do |storage|
admin_token = State.get! "admin_token"
storage.verify_admin(admin_token).should be_true
end
end
it "rejects non-admin token" do
with_storage do |storage|
user_token = State.get! "user_token"
storage.verify_admin(user_token).should be_false
end
end
it "updates user" do
with_storage do |storage|
storage.update_user "admin", "admin", "654321", true
token = storage.verify_user "admin", "654321"
admin_token = State.get! "admin_token"
token.should eq admin_token
end
end
it "logs user out" do
with_storage do |storage|
user_token = State.get! "user_token"
admin_token = State.get! "admin_token"
storage.logout user_token
storage.logout admin_token
storage.verify_token(user_token).should be_nil
storage.verify_token(admin_token).should be_nil
end
end
it "cleans up" do
with_storage do
true
end
State.reset
end
end

36
spec/util_spec.cr Normal file
View File

@@ -0,0 +1,36 @@
require "./spec_helper"
describe "compare_alphanumerically" do
it "sorts filenames with leading zeros correctly" do
ary = ["010.jpg", "001.jpg", "002.png"]
ary.sort! {|a, b|
compare_alphanumerically a, b
}
ary.should eq ["001.jpg", "002.png", "010.jpg"]
end
it "sorts filenames without leading zeros correctly" do
ary = ["10.jpg", "1.jpg", "0.png", "0100.jpg"]
ary.sort! {|a, b|
compare_alphanumerically a, b
}
ary.should eq ["0.png", "1.jpg", "10.jpg", "0100.jpg"]
end
# https://ux.stackexchange.com/a/95441
it "sorts like the stack exchange post" do
ary = ["2", "12", "200000", "1000000", "a", "a12", "b2", "text2",
"text2a", "text2a2", "text2a12", "text2ab", "text12", "text12a"]
ary.reverse.sort {|a, b|
compare_alphanumerically a, b
}.should eq ary
end
# https://github.com/hkalexling/Mango/issues/22
it "handles numbers larger than Int32" do
ary = ["14410155591588.jpg", "21410155591588.png", "104410155591588.jpg"]
ary.reverse.sort {|a, b|
compare_alphanumerically a, b
}.should eq ary
end
end

41
src/assets/lang_codes.csv Normal file
View File

@@ -0,0 +1,41 @@
Arabic,sa
Bengali,bd
Bulgarian,bg
Burmese,mm
Catalan,ct
Chinese (Simp),cn
Chinese (Trad),hk
Czech,cz
Danish,dk
Dutch,nl
English,gb
Filipino,ph
Finnish,fi
French,fr
German,de
Greek,gr
Hebrew,il
Hindi,in
Hungarian,hu
Indonesian,id
Italian,it
Japanese,jp
Korean,kr
Lithuanian,lt
Malay,my
Mongolian,mn
Other,
Persian,ir
Polish,pl
Portuguese (Br),br
Portuguese (Pt),pt
Romanian,ro
Russian,ru
Serbo-Croatian,rs
Spanish (Es),es
Spanish (LATAM),mx
Swedish,se
Thai,th
Turkish,tr
Ukrainian,ua
Vietnames,vn
1 Arabic sa
2 Bengali bd
3 Bulgarian bg
4 Burmese mm
5 Catalan ct
6 Chinese (Simp) cn
7 Chinese (Trad) hk
8 Czech cz
9 Danish dk
10 Dutch nl
11 English gb
12 Filipino ph
13 Finnish fi
14 French fr
15 German de
16 Greek gr
17 Hebrew il
18 Hindi in
19 Hungarian hu
20 Indonesian id
21 Italian it
22 Japanese jp
23 Korean kr
24 Lithuanian lt
25 Malay my
26 Mongolian mn
27 Other
28 Persian ir
29 Polish pl
30 Portuguese (Br) br
31 Portuguese (Pt) pt
32 Romanian ro
33 Russian ru
34 Serbo-Croatian rs
35 Spanish (Es) es
36 Spanish (LATAM) mx
37 Swedish se
38 Thai th
39 Turkish tr
40 Ukrainian ua
41 Vietnames vn

View File

@@ -15,7 +15,7 @@ class AuthHandler < Kemal::Handler
return env.redirect "/login"
end
if request_path_startswith env, ["/admin", "/api/admin"]
if request_path_startswith env, ["/admin", "/api/admin", "/download"]
unless @storage.verify_admin cookie.value
env.response.status_code = 403
end

View File

@@ -3,28 +3,33 @@ require "yaml"
class Config
include YAML::Serializable
@[YAML::Field(key: "port")]
property port : Int32 = 9000
@[YAML::Field(key: "library_path")]
property library_path : String = \
File.expand_path "~/mango/library", home: true
@[YAML::Field(key: "db_path")]
property db_path : String = \
File.expand_path "~/mango/mango.db", home: true
@[YAML::Field(key: "scan_interval_minutes")]
property scan_interval : Int32 = 5
@[YAML::Field(key: "log_level")]
property log_level : String = "info"
property mangadex = Hash(String, String|Int32).new
@[YAML::Field(ignore: true)]
@mangadex_defaults = {
"base_url" => "https://mangadex.org",
"api_url" => "https://mangadex.org/api",
"download_wait_seconds" => 5,
"download_retries" => 4,
"download_queue_db_path" => File.expand_path "~/mango/queue.db",
home: true
}
def self.load(path : String?)
path = "~/.config/mango/config.yml" if path.nil?
cfg_path = File.expand_path path, home: true
if File.exists? cfg_path
return self.from_yaml File.read cfg_path
config = self.from_yaml File.read cfg_path
config.fill_defaults
return config
end
puts "The config file #{cfg_path} does not exist." \
" Do you want mango to dump the default config there? [Y/n]"
@@ -33,6 +38,7 @@ class Config
abort "Aborting..."
end
default = self.allocate
default.fill_defaults
cfg_dir = File.dirname cfg_path
unless Dir.exists? cfg_dir
Dir.mkdir_p cfg_dir
@@ -41,4 +47,14 @@ class Config
puts "The config file has been created at #{cfg_path}."
default
end
def fill_defaults
{% for hash_name in ["mangadex"] %}
@{{hash_name.id}}_defaults.map do |k, v|
if @{{hash_name.id}}[k]?.nil?
@{{hash_name.id}}[k] = v
end
end
{% end %}
end
end

View File

@@ -8,8 +8,9 @@ class Context
property library : Library
property storage : Storage
property logger : MLogger
property queue : MangaDex::Queue
def initialize(@config, @logger, @library, @storage)
def initialize(@config, @logger, @library, @storage, @queue)
end
{% for lvl in LEVELS %}

View File

@@ -1,6 +1,8 @@
require "zip"
require "mime"
require "json"
require "uri"
require "./util"
struct Image
property data : Bytes
@@ -13,21 +15,42 @@ struct Image
end
class Entry
JSON.mapping zip_path: String, book_title: String, title: String, \
size: String, pages: Int32, cover_url: String
property zip_path : String, book_title : String, title : String,
size : String, pages : Int32, cover_url : String, id : String,
title_id : String, encoded_path : String, encoded_title : String,
mtime : Time
def initialize(path, @book_title)
def initialize(path, @book_title, @title_id, storage)
@zip_path = path
@encoded_path = URI.encode path
@title = File.basename path, File.extname path
@encoded_title = URI.encode @title
@size = (File.size path).humanize_bytes
@pages = Zip::File.new(path).entries
file = Zip::File.new path
@pages = file.entries
.select { |e|
["image/jpeg", "image/png"].includes? \
MIME.from_filename? e.filename
}
.size
@cover_url = "/api/page/#{@book_title}/#{title}/1"
file.close
@id = storage.get_id @zip_path, false
@cover_url = "/api/page/#{@title_id}/#{@id}/1"
@mtime = File.info(@zip_path).modification_time
end
def to_json(json : JSON::Builder)
json.object do
{% for str in ["zip_path", "book_title", "title", "size",
"cover_url", "id", "title_id", "encoded_path",
"encoded_title"] %}
json.field {{str}}, @{{str.id}}
{% end %}
json.field "pages" {json.number @pages}
json.field "mtime" {json.number @mtime.to_unix}
end
end
def read_page(page_num)
Zip::File.open @zip_path do |file|
page = file.entries
@@ -35,7 +58,9 @@ class Entry
["image/jpeg", "image/png"].includes? \
MIME.from_filename? e.filename
}
.sort { |a, b| a.filename <=> b.filename }
.sort { |a, b|
compare_alphanumerically a.filename, b.filename
}
.[page_num - 1]
page.open do |io|
slice = Bytes.new page.uncompressed_size
@@ -51,20 +76,114 @@ class Entry
end
class Title
JSON.mapping dir: String, entries: Array(Entry), title: String
property dir : String, parent_id : String, title_ids : Array(String),
entries : Array(Entry), title : String, id : String,
encoded_title : String, mtime : Time
def initialize(dir : String)
def initialize(dir : String, @parent_id, storage,
@logger : MLogger, @library : Library)
@dir = dir
@id = storage.get_id @dir, true
@title = File.basename dir
@entries = (Dir.entries dir)
.select { |path| [".zip", ".cbz"].includes? File.extname path }
.map { |path| Entry.new File.join(dir, path), @title }
.select { |e| e.pages > 0 }
.sort { |a, b| a.title <=> b.title }
@encoded_title = URI.encode @title
@title_ids = [] of String
@entries = [] of Entry
Dir.entries(dir).each do |fn|
next if fn.starts_with? "."
path = File.join dir, fn
if File.directory? path
title = Title.new path, @id, storage, @logger, library
next if title.entries.size == 0 && title.titles.size == 0
@library.title_hash[title.id] = title
@title_ids << title.id
next
end
if [".zip", ".cbz"].includes? File.extname path
next if !valid_zip path
entry = Entry.new path, @title, @id, storage
@entries << entry if entry.pages > 0
end
end
@title_ids.sort! do |a, b|
compare_alphanumerically @library.title_hash[a].title,
@library.title_hash[b].title
end
@entries.sort! do |a, b|
compare_alphanumerically a.title, b.title
end
mtimes = [File.info(dir).modification_time]
mtimes += @title_ids.map{|e| @library.title_hash[e].mtime}
mtimes += @entries.map{|e| e.mtime}
@mtime = mtimes.max
end
def get_entry(name)
@entries.find { |e| e.title == name }
def to_json(json : JSON::Builder)
json.object do
{% for str in ["dir", "title", "id", "encoded_title"] %}
json.field {{str}}, @{{str.id}}
{% end %}
json.field "mtime" {json.number @mtime.to_unix}
json.field "titles" do
json.raw self.titles.to_json
end
json.field "entries" do
json.raw @entries.to_json
end
json.field "parents" do
json.array do
self.parents.each do |title|
json.object do
json.field "title", title.title
json.field "id", title.id
end
end
end
end
end
end
def titles
@title_ids.map {|tid| @library.get_title! tid}
end
def parents
ary = [] of Title
tid = @parent_id
while !tid.empty?
title = @library.get_title! tid
ary << title
tid = title.parent_id
end
ary
end
def size
@entries.size + @title_ids.size
end
# When downloading from MangaDex, the zip/cbz file would not be valid
# before the download is completed. If we scan the zip file,
# Entry.new would throw, so we use this method to check before
# constructing Entry
private def valid_zip(path : String)
begin
file = Zip::File.new path
file.close
return true
rescue
@logger.warn "File #{path} is corrupted or is not a valid zip "\
"archive. Ignoring it."
return false
end
end
def get_entry(eid)
@entries.find { |e| e.id == eid }
end
# For backward backward compatibility with v0.1.0, we save entry titles
# instead of IDs in info.json
def save_progress(username, entry, page)
info = TitleInfo.new @dir
if info.progress[username]?.nil?
@@ -75,7 +194,7 @@ class Title
info.progress[username][entry] = page
info.save @dir
end
def load_progress(username, entry : String)
def load_progress(username, entry)
info = TitleInfo.new @dir
if info.progress[username]?.nil?
return 0
@@ -85,14 +204,15 @@ class Title
end
info.progress[username][entry]
end
def load_percetage(username, entry : String)
def load_percetage(username, entry)
info = TitleInfo.new @dir
page = load_progress username, entry
entry_obj = get_entry entry
return 0 if entry_obj.nil?
entry_obj = @entries.find{|e| e.title == entry}
return 0.0 if entry_obj.nil?
page / entry_obj.pages
end
def load_percetage(username)
return 0.0 if @entries.empty?
read_pages = total_pages = 0
@entries.each do |e|
read_pages += load_progress username, e.title
@@ -111,10 +231,7 @@ class TitleInfo
# { user1: { entry1: 10, entry2: 0 } }
include JSON::Serializable
@[JSON::Field(key: "comment")]
property comment = "Generated by Mango. DO NOT EDIT!"
@[JSON::Field(key: "progress")]
property progress : Hash(String, Hash(String, Int32))
def initialize(title_dir)
@@ -136,12 +253,14 @@ class TitleInfo
end
class Library
JSON.mapping dir: String, titles: Array(Title), scan_interval: Int32, logger: MLogger
property dir : String, title_ids : Array(String), scan_interval : Int32,
logger : MLogger, storage : Storage, title_hash : Hash(String, Title)
def initialize(@dir, @scan_interval, @logger)
def initialize(@dir, @scan_interval, @logger, @storage)
# explicitly initialize @titles to bypass the compiler check. it will
# be filled with actual Titles in the `scan` call below
@titles = [] of Title
@title_ids = [] of String
@title_hash = {} of String => Title
return scan if @scan_interval < 1
spawn do
@@ -149,13 +268,27 @@ class Library
start = Time.local
scan
ms = (Time.local - start).total_milliseconds
@logger.info "Scanned #{@titles.size} titles in #{ms}ms"
@logger.info "Scanned #{@title_ids.size} titles in #{ms}ms"
sleep @scan_interval * 60
end
end
end
def get_title(name)
@titles.find { |t| t.title == name }
def titles
@title_ids.map {|tid| self.get_title!(tid) }
end
def to_json(json : JSON::Builder)
json.object do
json.field "dir", @dir
json.field "titles" do
json.raw self.titles.to_json
end
end
end
def get_title(tid)
@title_hash[tid]?
end
def get_title!(tid)
@title_hash[tid]
end
def scan
unless Dir.exists? @dir
@@ -163,11 +296,18 @@ class Library
"Attempting to create it"
Dir.mkdir_p @dir
end
@titles = (Dir.entries @dir)
.select { |path| File.directory? File.join @dir, path }
.map { |path| Title.new File.join @dir, path }
.select { |title| !title.entries.empty? }
@title_ids.clear
(Dir.entries @dir)
.select { |fn| !fn.starts_with? "." }
.map { |fn| File.join @dir, fn }
.select { |path| File.directory? path }
.map { |path| Title.new path, "", @storage, @logger, self }
.select { |title| !(title.entries.empty? && title.titles.empty?) }
.sort { |a, b| a.title <=> b.title }
.each do |title|
@title_hash[title.id] = title
@title_ids << title.id
end
@logger.debug "Scan completed"
@logger.debug "Scanned library: \n#{self.to_pretty_json}"
end
end

203
src/mangadex/api.cr Normal file
View File

@@ -0,0 +1,203 @@
require "http/client"
require "json"
require "csv"
macro string_properties (names)
{% for name in names %}
property {{name.id}} = ""
{% end %}
end
macro parse_strings_from_json (names)
{% for name in names %}
@{{name.id}} = obj[{{name}}].as_s
{% end %}
end
module MangaDex
class Chapter
string_properties ["lang_code", "title", "volume", "chapter"]
property manga : Manga
property time = Time.local
property id : String
property full_title = ""
property language = ""
property pages = [] of {String, String} # filename, url
property groups = [] of {Int32, String} # group_id, group_name
def initialize(@id, json_obj : JSON::Any, @manga, lang :
Hash(String, String))
self.parse_json json_obj, lang
end
def to_info_json
JSON.build do |json|
json.object do
{% for name in ["id", "title", "volume", "chapter",
"language", "full_title"] %}
json.field {{name}}, @{{name.id}}
{% end %}
json.field "time", @time.to_unix.to_s
json.field "manga_title", @manga.title
json.field "manga_id", @manga.id
json.field "groups" do
json.object do
@groups.each do |gid, gname|
json.field gname, gid
end
end
end
end
end
end
def parse_json(obj, lang)
begin
parse_strings_from_json ["lang_code", "title", "volume",
"chapter"]
language = lang[@lang_code]?
@language = language if language
@time = Time.unix obj["timestamp"].as_i
suffixes = ["", "_2", "_3"]
suffixes.each do |s|
gid = obj["group_id#{s}"].as_i
next if gid == 0
gname = obj["group_name#{s}"].as_s
@groups << {gid, gname}
end
@full_title = @title
unless @chapter.empty?
@full_title = "Ch.#{@chapter} " + @full_title
end
unless @volume.empty?
@full_title = "Vol.#{@volume} " + @full_title
end
rescue e
raise "failed to parse json: #{e}"
end
end
end
class Manga
string_properties ["cover_url", "description", "title", "author",
"artist"]
property chapters = [] of Chapter
property id : String
def initialize(@id, json_obj : JSON::Any)
self.parse_json json_obj
end
def to_info_json(with_chapters = true)
JSON.build do |json|
json.object do
{% for name in ["id", "title", "description",
"author", "artist", "cover_url"] %}
json.field {{name}}, @{{name.id}}
{% end %}
if with_chapters
json.field "chapters" do
json.array do
@chapters.each do |c|
json.raw c.to_info_json
end
end
end
end
end
end
end
def parse_json(obj)
begin
parse_strings_from_json ["cover_url", "description", "title",
"author", "artist"]
rescue e
raise "failed to parse json: #{e}"
end
end
end
class API
def initialize(@base_url = "https://mangadex.org/api/")
@lang = {} of String => String
CSV.each_row {{read_file "src/assets/lang_codes.csv"}} do |row|
@lang[row[1]] = row[0]
end
end
def get(url)
headers = HTTP::Headers {
"User-agent" => "Mangadex.cr"
}
res = HTTP::Client.get url, headers
raise "Failed to get #{url}. [#{res.status_code}] "\
"#{res.status_message}" if !res.success?
JSON.parse res.body
end
def get_manga(id)
obj = self.get File.join @base_url, "manga/#{id}"
if obj["status"]? != "OK"
raise "Expecting `OK` in the `status` field. " \
"Got `#{obj["status"]?}`"
end
begin
manga = Manga.new id, obj["manga"]
obj["chapter"].as_h.map do |k, v|
chapter = Chapter.new k, v, manga, @lang
manga.chapters << chapter
end
return manga
rescue
raise "Failed to parse JSON"
end
end
def get_chapter(chapter : Chapter)
obj = self.get File.join @base_url, "chapter/#{chapter.id}"
if obj["status"]? == "external"
raise "This chapter is hosted on an external site " \
"#{obj["external"]?}, and Mango does not support " \
"external chapters."
end
if obj["status"]? != "OK"
raise "Expecting `OK` in the `status` field. " \
"Got `#{obj["status"]?}`"
end
begin
server = obj["server"].as_s
hash = obj["hash"].as_s
chapter.pages = obj["page_array"].as_a.map do |fn|
{
fn.as_s,
"#{server}#{hash}/#{fn.as_s}"
}
end
rescue
raise "Failed to parse JSON"
end
end
def get_chapter(id : String)
obj = self.get File.join @base_url, "chapter/#{id}"
if obj["status"]? == "external"
raise "This chapter is hosted on an external site " \
"#{obj["external"]?}, and Mango does not support " \
"external chapters."
end
if obj["status"]? != "OK"
raise "Expecting `OK` in the `status` field. " \
"Got `#{obj["status"]?}`"
end
manga_id = ""
begin
manga_id = obj["manga_id"].as_i.to_s
rescue
raise "Failed to parse JSON"
end
manga = self.get_manga manga_id
chapter = manga.chapters.find {|c| c.id == id}.not_nil!
self.get_chapter chapter
return chapter
end
end
end

374
src/mangadex/downloader.cr Normal file
View File

@@ -0,0 +1,374 @@
require "./api"
require "sqlite3"
module MangaDex
class PageJob
property success = false
property url : String
property filename : String
property writer : Zip::Writer
property tries_remaning : Int32
def initialize(@url, @filename, @writer, @tries_remaning)
end
end
enum JobStatus
Pending # 0
Downloading # 1
Error # 2
Completed # 3
MissingPages # 4
end
struct Job
property id : String
property manga_id : String
property title : String
property manga_title : String
property status : JobStatus
property status_message : String = ""
property pages : Int32 = 0
property success_count : Int32 = 0
property fail_count : Int32 = 0
property time : Time
def parse_query_result(res : DB::ResultSet)
@id = res.read String
@manga_id = res.read String
@title = res.read String
@manga_title = res.read String
status = res.read Int32
@status_message = res.read String
@pages = res.read Int32
@success_count = res.read Int32
@fail_count = res.read Int32
time = res.read Int64
@status = JobStatus.new status
@time = Time.unix_ms time
end
# Raises if the result set does not contain the correct set of columns
def self.from_query_result(res : DB::ResultSet)
job = Job.allocate
job.parse_query_result res
return job
end
def initialize(@id, @manga_id, @title, @manga_title, @status, @time)
end
def to_json(json)
json.object do
{% for name in ["id", "manga_id", "title", "manga_title",
"status_message"] %}
json.field {{name}}, @{{name.id}}
{% end %}
{% for name in ["pages", "success_count", "fail_count"] %}
json.field {{name}} do
json.number @{{name.id}}
end
{% end %}
json.field "status", @status.to_s
json.field "time" do
json.number @time.to_unix_ms
end
end
end
end
class Queue
property downloader : Downloader?
def initialize(@path : String, @logger : MLogger)
dir = File.dirname path
unless Dir.exists? dir
@logger.info "The queue DB directory #{dir} does not exist. " \
"Attepmting to create it"
Dir.mkdir_p dir
end
DB.open "sqlite3://#{@path}" do |db|
begin
db.exec "create table if not exists queue " \
"(id text, manga_id text, title text, manga_title " \
"text, status integer, status_message text, " \
"pages integer, success_count integer, " \
"fail_count integer, time integer)"
db.exec "create unique index if not exists id_idx " \
"on queue (id)"
db.exec "create index if not exists manga_id_idx " \
"on queue (manga_id)"
db.exec "create index if not exists status_idx " \
"on queue (status)"
rescue e
@logger.error "Error when checking tables in DB: #{e}"
raise e
end
end
end
# Returns the earliest job in queue or nil if the job cannot be parsed.
# Returns nil if queue is empty
def pop
job = nil
DB.open "sqlite3://#{@path}" do |db|
begin
db.query_one "select * from queue where status = 0 "\
"or status = 1 order by time limit 1" do |res|
job = Job.from_query_result res
end
rescue
end
end
return job
end
# Push an array of jobs into the queue, and return the number of jobs
# inserted. Any job already exists in the queue will be ignored.
def push(jobs : Array(Job))
start_count = self.count
DB.open "sqlite3://#{@path}" do |db|
jobs.each do |job|
db.exec "insert or ignore into queue values "\
"(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
job.id, job.manga_id, job.title, job.manga_title,
job.status.to_i, job.status_message, job.pages,
job.success_count, job.fail_count, job.time.to_unix_ms
end
end
self.count - start_count
end
def reset(id : String)
DB.open "sqlite3://#{@path}" do |db|
db.exec "update queue set status = 0, status_message = '', " \
"pages = 0, success_count = 0, fail_count = 0 " \
"where id = (?)", id
end
end
def reset (job : Job)
self.reset job.id
end
# Reset all failed tasks (missing pages and error)
def reset
DB.open "sqlite3://#{@path}" do |db|
db.exec "update queue set status = 0, status_message = '', " \
"pages = 0, success_count = 0, fail_count = 0 " \
"where status = 2 or status = 4"
end
end
def delete(id : String)
DB.open "sqlite3://#{@path}" do |db|
db.exec "delete from queue where id = (?)", id
end
end
def delete(job : Job)
self.delete job.id
end
def delete_status(status : JobStatus)
DB.open "sqlite3://#{@path}" do |db|
db.exec "delete from queue where status = (?)", status.to_i
end
end
def count_status(status : JobStatus)
DB.open "sqlite3://#{@path}" do |db|
return db.query_one "select count(*) from queue where "\
"status = (?)", status.to_i, as: Int32
end
end
def count
DB.open "sqlite3://#{@path}" do |db|
return db.query_one "select count(*) from queue", as: Int32
end
end
def set_status(status : JobStatus, job : Job)
DB.open "sqlite3://#{@path}" do |db|
db.exec "update queue set status = (?) where id = (?)",
status.to_i, job.id
end
end
def get_all
jobs = [] of Job
DB.open "sqlite3://#{@path}" do |db|
jobs = db.query_all "select * from queue order by time", do |rs|
Job.from_query_result rs
end
end
return jobs
end
def add_success(job : Job)
DB.open "sqlite3://#{@path}" do |db|
db.exec "update queue set success_count = success_count + 1 " \
"where id = (?)", job.id
end
end
def add_fail(job : Job)
DB.open "sqlite3://#{@path}" do |db|
db.exec "update queue set fail_count = fail_count + 1 " \
"where id = (?)", job.id
end
end
def set_pages(pages : Int32, job : Job)
DB.open "sqlite3://#{@path}" do |db|
db.exec "update queue set pages = (?), success_count = 0, " \
"fail_count = 0 where id = (?)", pages, job.id
end
end
def add_message(msg : String, job : Job)
DB.open "sqlite3://#{@path}" do |db|
db.exec "update queue set status_message = " \
"status_message || (?) || (?) where id = (?)",
"\n", msg, job.id
end
end
def pause
@downloader.not_nil!.stopped = true
end
def resume
@downloader.not_nil!.stopped = false
end
def paused?
@downloader.not_nil!.stopped
end
end
class Downloader
property stopped = false
@downloading = false
def initialize(@queue : Queue, @api : API, @library_path : String,
@wait_seconds : Int32, @retries : Int32,
@logger : MLogger)
@queue.downloader = self
spawn do
loop do
sleep 1.second
next if @stopped || @downloading
begin
job = @queue.pop
next if job.nil?
download job
rescue e
@logger.error e
end
end
end
end
private def download(job : Job)
@downloading = true
@queue.set_status JobStatus::Downloading, job
begin
chapter = @api.get_chapter(job.id)
rescue e
@logger.error e
@queue.set_status JobStatus::Error, job
unless e.message.nil?
@queue.add_message e.message.not_nil!, job
end
@downloading = false
return
end
@queue.set_pages chapter.pages.size, job
lib_dir = @library_path
manga_dir = File.join lib_dir, chapter.manga.title
unless File.exists? manga_dir
Dir.mkdir_p manga_dir
end
zip_path = File.join manga_dir, "#{job.title}.cbz"
# Find the number of digits needed to store the number of pages
len = Math.log10(chapter.pages.size).to_i + 1
writer = Zip::Writer.new zip_path
# Create a buffered channel. It works as an FIFO queue
channel = Channel(PageJob).new chapter.pages.size
spawn do
chapter.pages.each_with_index do |tuple, i|
fn, url = tuple
ext = File.extname fn
fn = "#{i.to_s.rjust len, '0'}#{ext}"
page_job = PageJob.new url, fn, writer, @retries
@logger.debug "Downloading #{url}"
loop do
sleep @wait_seconds.seconds
download_page page_job
break if page_job.success ||
page_job.tries_remaning <= 0
page_job.tries_remaning -= 1
@logger.warn "Failed to download page #{url}. " \
"Retrying... Remaining retries: " \
"#{page_job.tries_remaning}"
end
channel.send page_job
end
end
spawn do
page_jobs = [] of PageJob
chapter.pages.size.times do
page_job = channel.receive
@logger.debug "[#{page_job.success ? "success" : "failed"}] " \
"#{page_job.url}"
page_jobs << page_job
if page_job.success
@queue.add_success job
else
@queue.add_fail job
msg = "Failed to download page #{page_job.url}"
@queue.add_message msg, job
@logger.error msg
end
end
fail_count = page_jobs.select{|j| !j.success}.size
@logger.debug "Download completed. "\
"#{fail_count}/#{page_jobs.size} failed"
writer.close
@logger.debug "cbz File created at #{zip_path}"
if fail_count == 0
@queue.set_status JobStatus::Completed, job
else
@queue.set_status JobStatus::MissingPages, job
end
@downloading = false
end
end
private def download_page(job : PageJob)
@logger.debug "downloading #{job.url}"
headers = HTTP::Headers {
"User-agent" => "Mangadex.cr"
}
begin
HTTP::Client.get job.url, headers do |res|
unless res.success?
raise "Failed to download page #{job.url}. " \
"[#{res.status_code}] #{res.status_message}"
end
job.writer.add job.filename, res.body_io
end
job.success = true
rescue e
@logger.error e
job.success = false
end
end
end
end

View File

@@ -1,8 +1,9 @@
require "./server"
require "./context"
require "./mangadex/*"
require "option_parser"
VERSION = "0.1.0"
VERSION = "0.2.4"
config_path = nil
@@ -25,10 +26,16 @@ end
config = Config.load config_path
logger = MLogger.new config
library = Library.new config.library_path, config.scan_interval, logger
storage = Storage.new config.db_path, logger
library = Library.new config.library_path, config.scan_interval, logger, storage
queue = MangaDex::Queue.new config.mangadex["download_queue_db_path"].to_s,
logger
api = MangaDex::API.new config.mangadex["api_url"].to_s
downloader = MangaDex::Downloader.new queue, api, config.library_path,
config.mangadex["download_wait_seconds"].to_i,
config.mangadex["download_retries"].to_i, logger
context = Context.new config, logger, library, storage
context = Context.new config, logger, library, storage, queue
server = Server.new context
server.start

108
src/routes/admin.cr Normal file
View File

@@ -0,0 +1,108 @@
require "./router"
class AdminRouter < Router
def setup
get "/admin" do |env|
layout "admin"
end
get "/admin/user" do |env|
users = @context.storage.list_users
username = get_username env
layout "user"
end
get "/admin/user/edit" do |env|
username = env.params.query["username"]?
admin = env.params.query["admin"]?
if admin
admin = admin == "true"
end
error = env.params.query["error"]?
current_user = get_username env
new_user = username.nil? && admin.nil?
layout "user-edit"
end
post "/admin/user/edit" do |env|
# creating new user
begin
username = env.params.body["username"]
password = env.params.body["password"]
# if `admin` is unchecked, the body hash
# would not contain `admin`
admin = !env.params.body["admin"]?.nil?
if username.size < 3
raise "Username should contain at least 3 characters"
end
if (username =~ /^[A-Za-z0-9_]+$/).nil?
raise "Username should contain alphanumeric characters "\
"and underscores only"
end
if password.size < 6
raise "Password should contain at least 6 characters"
end
if (password =~ /^[[:ascii:]]+$/).nil?
raise "password should contain ASCII characters only"
end
@context.storage.new_user username, password, admin
env.redirect "/admin/user"
rescue e
@context.error e
redirect_url = URI.new \
path: "/admin/user/edit",\
query: hash_to_query({"error" => e.message})
env.redirect redirect_url.to_s
end
end
post "/admin/user/edit/:original_username" do |env|
# editing existing user
begin
username = env.params.body["username"]
password = env.params.body["password"]
# if `admin` is unchecked, the body
# hash would not contain `admin`
admin = !env.params.body["admin"]?.nil?
original_username = env.params.url["original_username"]
if username.size < 3
raise "Username should contain at least 3 characters"
end
if (username =~ /^[A-Za-z0-9_]+$/).nil?
raise "Username should contain alphanumeric characters "\
"and underscores only"
end
if password.size != 0
if password.size < 6
raise "Password should contain at least 6 characters"
end
if (password =~ /^[[:ascii:]]+$/).nil?
raise "password should contain ASCII characters only"
end
end
@context.storage.update_user \
original_username, username, password, admin
env.redirect "/admin/user"
rescue e
@context.error e
redirect_url = URI.new \
path: "/admin/user/edit",\
query: hash_to_query({"username" => original_username, \
"admin" => admin, "error" => e.message})
env.redirect redirect_url.to_s
end
end
get "/admin/downloads" do |env|
base_url = @context.config.mangadex["base_url"];
layout "download-manager"
end
end
end

180
src/routes/api.cr Normal file
View File

@@ -0,0 +1,180 @@
require "./router"
require "../mangadex/*"
class APIRouter < Router
def setup
get "/api/page/:tid/:eid/:page" do |env|
begin
tid = env.params.url["tid"]
eid = env.params.url["eid"]
page = env.params.url["page"].to_i
title = @context.library.get_title tid
raise "Title ID `#{tid}` not found" if title.nil?
entry = title.get_entry eid
raise "Entry ID `#{eid}` of `#{title.title}` not found" if \
entry.nil?
img = entry.read_page page
raise "Failed to load page #{page} of " \
"`#{title.title}/#{entry.title}`" if img.nil?
send_img env, img
rescue e
@context.error e
env.response.status_code = 500
e.message
end
end
get "/api/book/:tid" do |env|
begin
tid = env.params.url["tid"]
title = @context.library.get_title tid
raise "Title ID `#{tid}` not found" if title.nil?
send_json env, title.to_json
rescue e
@context.error e
env.response.status_code = 500
e.message
end
end
get "/api/book" do |env|
send_json env, @context.library.to_json
end
post "/api/admin/scan" do |env|
start = Time.utc
@context.library.scan
ms = (Time.utc - start).total_milliseconds
send_json env, {
"milliseconds" => ms,
"titles" => @context.library.titles.size
}.to_json
end
post "/api/admin/user/delete/:username" do |env|
begin
username = env.params.url["username"]
@context.storage.delete_user username
rescue e
@context.error e
send_json env, {
"success" => false,
"error" => e.message
}.to_json
else
send_json env, {"success" => true}.to_json
end
end
post "/api/progress/:title/:entry/:page" do |env|
begin
username = get_username env
title = (@context.library.get_title env.params.url["title"])
.not_nil!
entry = (title.get_entry env.params.url["entry"]).not_nil!
page = env.params.url["page"].to_i
raise "incorrect page value" if page < 0 || page > entry.pages
title.save_progress username, entry.title, page
rescue e
@context.error e
send_json env, {
"success" => false,
"error" => e.message
}.to_json
else
send_json env, {"success" => true}.to_json
end
end
get "/api/admin/mangadex/manga/:id" do |env|
begin
id = env.params.url["id"]
api = MangaDex::API.new \
@context.config.mangadex["api_url"].to_s
manga = api.get_manga id
send_json env, manga.to_info_json
rescue e
@context.error e
send_json env, {"error" => e.message}.to_json
end
end
post "/api/admin/mangadex/download" do |env|
begin
chapters = env.params.json["chapters"].as(Array).map{|c| c.as_h}
jobs = chapters.map {|chapter|
MangaDex::Job.new(
chapter["id"].as_s,
chapter["manga_id"].as_s,
chapter["full_title"].as_s,
chapter["manga_title"].as_s,
MangaDex::JobStatus::Pending,
Time.unix chapter["time"].as_s.to_i
)
}
inserted_count = @context.queue.push jobs
send_json env, {
"success": inserted_count,
"fail": jobs.size - inserted_count
}.to_json
rescue e
@context.error e
send_json env, {"error" => e.message}.to_json
end
end
get "/api/admin/mangadex/queue" do |env|
begin
jobs = @context.queue.get_all
send_json env, {
"jobs" => jobs,
"paused" => @context.queue.paused?,
"success" => true
}.to_json
rescue e
send_json env, {
"success" => false,
"error" => e.message
}.to_json
end
end
post "/api/admin/mangadex/queue/:action" do |env|
begin
action = env.params.url["action"]
id = env.params.query["id"]?
case action
when "delete"
if id.nil?
@context.queue.delete_status MangaDex::JobStatus::Completed
else
@context.queue.delete id
end
when "retry"
if id.nil?
@context.queue.reset
else
@context.queue.reset id
end
when "pause"
@context.queue.pause
when "resume"
@context.queue.resume
else
raise "Unknown queue action #{action}"
end
send_json env, {"success" => true}.to_json
rescue e
send_json env, {
"success" => false,
"error" => e.message
}.to_json
end
end
end
end

63
src/routes/main.cr Normal file
View File

@@ -0,0 +1,63 @@
require "./router"
class MainRouter < Router
def setup
get "/login" do |env|
render "src/views/login.ecr"
end
get "/logout" do |env|
begin
cookie = env.request.cookies
.find { |c| c.name == "token" }.not_nil!
@context.storage.logout cookie.value
rescue e
@context.error "Error when attempting to log out: #{e}"
ensure
env.redirect "/login"
end
end
post "/login" do |env|
begin
username = env.params.body["username"]
password = env.params.body["password"]
token = @context.storage.verify_user(username, password)
.not_nil!
cookie = HTTP::Cookie.new "token", token
cookie.expires = Time.local.shift years: 1
env.response.cookies << cookie
env.redirect "/"
rescue
env.redirect "/login"
end
end
get "/" do |env|
titles = @context.library.titles
username = get_username env
percentage = titles.map &.load_percetage username
layout "index"
end
get "/book/:title" do |env|
begin
title = (@context.library.get_title env.params.url["title"])
.not_nil!
username = get_username env
percentage = title.entries.map { |e|
title.load_percetage username, e.title }
layout "title"
rescue e
@context.error e
env.response.status_code = 404
end
end
get "/download" do |env|
base_url = @context.config.mangadex["base_url"];
layout "download"
end
end
end

58
src/routes/reader.cr Normal file
View File

@@ -0,0 +1,58 @@
require "./router"
class ReaderRouter < Router
def setup
get "/reader/:title/:entry" do |env|
begin
title = (@context.library.get_title env.params.url["title"])
.not_nil!
entry = (title.get_entry env.params.url["entry"]).not_nil!
# load progress
username = get_username env
page = title.load_progress username, entry.title
# we go back 2 * `IMGS_PER_PAGE` pages. the infinite scroll
# library perloads a few pages in advance, and the user
# might not have actually read them
page = [page - 2 * IMGS_PER_PAGE, 1].max
env.redirect "/reader/#{title.id}/#{entry.id}/#{page}"
rescue e
@context.error e
env.response.status_code = 404
end
end
get "/reader/:title/:entry/:page" do |env|
begin
title = (@context.library.get_title env.params.url["title"])
.not_nil!
entry = (title.get_entry env.params.url["entry"]).not_nil!
page = env.params.url["page"].to_i
raise "" if page > entry.pages || page <= 0
# save progress
username = get_username env
title.save_progress username, entry.title, page
pages = (page...[entry.pages + 1, page + IMGS_PER_PAGE].min)
urls = pages.map { |idx|
"/api/page/#{title.id}/#{entry.id}/#{idx}" }
reader_urls = pages.map { |idx|
"/reader/#{title.id}/#{entry.id}/#{idx}" }
next_page = page + IMGS_PER_PAGE
next_url = next_page > entry.pages ? nil :
"/reader/#{title.id}/#{entry.id}/#{next_page}"
exit_url = "/book/#{title.id}"
next_entry = title.next_entry entry
next_entry_url = next_entry.nil? ? nil : \
"/reader/#{title.id}/#{next_entry.id}"
render "src/views/reader.ecr"
rescue e
@context.error e
env.response.status_code = 404
end
end
end
end

6
src/routes/router.cr Normal file
View File

@@ -0,0 +1,6 @@
require "../context"
class Router
def initialize(@context : Context)
end
end

View File

@@ -4,312 +4,36 @@ require "./auth_handler"
require "./static_handler"
require "./log_handler"
require "./util"
require "./routes/*"
class Server
def initialize(@context : Context)
error 403 do |env|
message = "You are not authorized to visit #{env.request.path}"
message = "HTTP 403: You are not authorized to visit " \
"#{env.request.path}"
layout "message"
end
error 404 do |env|
message = "HTTP 404: Mango cannot find the page #{env.request.path}"
layout "message"
end
error 500 do |env|
message = "HTTP 500: Internal server error. Please try again later."
layout "message"
end
get "/" do |env|
titles = @context.library.titles
username = get_username env
percentage = titles.map &.load_percetage username
layout "index"
end
get "/book/:title" do |env|
begin
title = (@context.library.get_title env.params.url["title"])
.not_nil!
username = get_username env
percentage = title.entries.map { |e|
title.load_percetage username, e.title }
layout "title"
rescue e
@context.error e
env.response.status_code = 404
end
end
get "/admin" do |env|
layout "admin"
end
get "/admin/user" do |env|
users = @context.storage.list_users
username = get_username env
layout "user"
end
get "/admin/user/edit" do |env|
username = env.params.query["username"]?
admin = env.params.query["admin"]?
if admin
admin = admin == "true"
end
error = env.params.query["error"]?
current_user = get_username env
new_user = username.nil? && admin.nil?
layout "user-edit"
end
post "/admin/user/edit" do |env|
# creating new user
begin
username = env.params.body["username"]
password = env.params.body["password"]
# if `admin` is unchecked, the body hash
# would not contain `admin`
admin = !env.params.body["admin"]?.nil?
if username.size < 3
raise "Username should contain at least 3 characters"
end
if (username =~ /^[A-Za-z0-9_]+$/).nil?
raise "Username should contain alphanumeric characters "\
"and underscores only"
end
if password.size < 6
raise "Password should contain at least 6 characters"
end
if (password =~ /^[[:ascii:]]+$/).nil?
raise "password should contain ASCII characters only"
end
@context.storage.new_user username, password, admin
env.redirect "/admin/user"
rescue e
@context.error e
redirect_url = URI.new \
path: "/admin/user/edit",\
query: hash_to_query({"error" => e.message})
env.redirect redirect_url.to_s
end
end
post "/admin/user/edit/:original_username" do |env|
# editing existing user
begin
username = env.params.body["username"]
password = env.params.body["password"]
# if `admin` is unchecked, the body
# hash would not contain `admin`
admin = !env.params.body["admin"]?.nil?
original_username = env.params.url["original_username"]
if username.size < 3
raise "Username should contain at least 3 characters"
end
if (username =~ /^[A-Za-z0-9_]+$/).nil?
raise "Username should contain alphanumeric characters "\
"and underscores only"
end
if password.size != 0
if password.size < 6
raise "Password should contain at least 6 characters"
end
if (password =~ /^[[:ascii:]]+$/).nil?
raise "password should contain ASCII characters only"
end
end
@context.storage.update_user \
original_username, username, password, admin
env.redirect "/admin/user"
rescue e
@context.error e
redirect_url = URI.new \
path: "/admin/user/edit",\
query: hash_to_query({"username" => original_username, \
"admin" => admin, "error" => e.message})
env.redirect redirect_url.to_s
end
end
get "/reader/:title/:entry" do |env|
begin
title = (@context.library.get_title env.params.url["title"])
.not_nil!
entry = (title.get_entry env.params.url["entry"]).not_nil!
# load progress
username = get_username env
page = title.load_progress username, entry.title
# we go back 2 * `IMGS_PER_PAGE` pages. the infinite scroll
# library perloads a few pages in advance, and the user
# might not have actually read them
page = [page - 2 * IMGS_PER_PAGE, 1].max
env.redirect "/reader/#{title.title}/#{entry.title}/#{page}"
rescue e
@context.error e
env.response.status_code = 404
end
end
get "/reader/:title/:entry/:page" do |env|
begin
title = (@context.library.get_title env.params.url["title"])
.not_nil!
entry = (title.get_entry env.params.url["entry"]).not_nil!
page = env.params.url["page"].to_i
raise "" if page > entry.pages || page <= 0
# save progress
username = get_username env
title.save_progress username, entry.title, page
pages = (page...[entry.pages + 1, page + IMGS_PER_PAGE].min)
urls = pages.map { |idx|
"/api/page/#{title.title}/#{entry.title}/#{idx}" }
reader_urls = pages.map { |idx|
"/reader/#{title.title}/#{entry.title}/#{idx}" }
next_page = page + IMGS_PER_PAGE
next_url = next_page > entry.pages ? nil :
"/reader/#{title.title}/#{entry.title}/#{next_page}"
exit_url = "/book/#{title.title}"
next_entry = title.next_entry entry
next_entry_url = next_entry.nil? ? nil : \
"/reader/#{title.title}/#{next_entry.title}"
render "src/views/reader.ecr"
rescue e
@context.error e
env.response.status_code = 404
end
end
get "/login" do |env|
render "src/views/login.ecr"
end
get "/logout" do |env|
begin
cookie = env.request.cookies
.find { |c| c.name == "token" }.not_nil!
@context.storage.logout cookie.value
rescue e
@context.error "Error when attempting to log out: #{e}"
ensure
env.redirect "/login"
end
end
post "/login" do |env|
begin
username = env.params.body["username"]
password = env.params.body["password"]
token = @context.storage.verify_user(username, password)
.not_nil!
cookie = HTTP::Cookie.new "token", token
env.response.cookies << cookie
env.redirect "/"
rescue
env.redirect "/login"
end
end
get "/api/page/:title/:entry/:page" do |env|
begin
title = env.params.url["title"]
entry = env.params.url["entry"]
page = env.params.url["page"].to_i
t = @context.library.get_title title
raise "Title `#{title}` not found" if t.nil?
e = t.get_entry entry
raise "Entry `#{entry}` of `#{title}` not found" if e.nil?
img = e.read_page page
raise "Failed to load page #{page} of `#{title}/#{entry}`"\
if img.nil?
send_img env, img
rescue e
@context.error e
env.response.status_code = 500
e.message
end
end
get "/api/book/:title" do |env|
begin
title = env.params.url["title"]
t = @context.library.get_title title
raise "Title `#{title}` not found" if t.nil?
send_json env, t.to_json
rescue e
@context.error e
env.response.status_code = 500
e.message
end
end
get "/api/book" do |env|
send_json env, @context.library.to_json
end
post "/api/admin/scan" do |env|
start = Time.utc
@context.library.scan
ms = (Time.utc - start).total_milliseconds
send_json env, {
"milliseconds" => ms,
"titles" => @context.library.titles.size
}.to_json
end
post "/api/admin/user/delete/:username" do |env|
begin
username = env.params.url["username"]
@context.storage.delete_user username
rescue e
@context.error e
send_json env, {
"success" => false,
"error" => e.message
}.to_json
else
send_json env, {"success" => true}.to_json
end
end
post "/api/progress/:title/:entry/:page" do |env|
begin
username = get_username env
title = (@context.library.get_title env.params.url["title"])
.not_nil!
entry = (title.get_entry env.params.url["entry"]).not_nil!
page = env.params.url["page"].to_i
raise "incorrect page value" if page < 0 || page > entry.pages
title.save_progress username, entry.title, page
rescue e
@context.error e
send_json env, {
"success" => false,
"error" => e.message
}.to_json
else
send_json env, {"success" => true}.to_json
end
end
MainRouter.new(@context).setup
AdminRouter.new(@context).setup
ReaderRouter.new(@context).setup
APIRouter.new(@context).setup
Kemal.config.logging = false
add_handler LogHandler.new @context.logger
add_handler AuthHandler.new @context.storage
{% if flag?(:release) %}
# when building for relase, embed the static files in binary
@context.debug "We are in release mode. Using embeded static files."
@context.debug "We are in release mode. Using embedded static files."
serve_static false
add_handler StaticHandler.new
{% end %}

View File

@@ -1,14 +1,17 @@
require "baked_file_system"
require "kemal"
require "gzip"
require "./util"
class FS
extend BakedFileSystem
{% if read_file? "./dist" %}
bake_folder "../dist"
{% else %}
bake_folder "../public"
{% if flag?(:release) %}
{% if read_file? "#{__DIR__}/../dist/favicon.ico" %}
{% puts "baking ../dist" %}
bake_folder "../dist"
{% else %}
{% puts "baking ../public" %}
bake_folder "../public"
{% end %}
{% end %}
end

View File

@@ -12,7 +12,7 @@ def verify_password(hash, pw)
end
def random_str
Base64.strict_encode UUID.random().to_s
UUID.random.to_s.gsub "-", ""
end
class Storage
@@ -25,10 +25,18 @@ class Storage
end
DB.open "sqlite3://#{path}" do |db|
begin
# We create the `ids` table first. even if the uses has an
# early version installed and has the `user` table only,
# we will still be able to create `ids`
db.exec "create table ids" \
"(path text, id text, is_title integer)"
db.exec "create unique index path_idx on ids (path)"
db.exec "create unique index id_idx on ids (id)"
db.exec "create table users" \
"(username text, password text, token text, admin integer)"
rescue e
unless e.message == "table users already exists"
unless e.message.not_nil!.ends_with? "already exists"
@logger.fatal "Error when checking tables in DB: #{e}"
raise e
end
@@ -147,4 +155,23 @@ class Storage
end
end
end
def get_id(path, is_title)
DB.open "sqlite3://#{@path}" do |db|
begin
id = db.query_one "select id from ids where path = (?)",
path, as: {String}
return id
rescue
id = random_str
db.exec "insert into ids values (?, ?, ?)", path, id,
is_title ? 1 : 0
return id
end
end
end
def to_json(json : JSON::Builder)
json.string self
end
end

View File

@@ -1,3 +1,5 @@
require "big"
IMGS_PER_PAGE = 5
macro layout(name)
@@ -9,7 +11,7 @@ macro send_img(env, img)
end
macro get_username(env)
# if the request gets here, its has gone through the auth handler, and
# if the request gets here, it has gone through the auth handler, and
# we can be sure that a valid token exists, so we can use not_nil! here
cookie = {{env}}.request.cookies.find { |c| c.name == "token" }.not_nil!
(@context.storage.verify_token cookie.value).not_nil!
@@ -32,3 +34,40 @@ def request_path_startswith(env, ary)
end
return false
end
def is_numeric(str)
/^\d+/.match(str) != nil
end
def split_by_alphanumeric(str)
arr = [] of String
str.scan(/([^\d\n\r]*)(\d*)([^\d\n\r]*)/) do |match|
arr += match.captures.select{|s| s != ""}
end
arr
end
def compare_alphanumerically(c, d)
is_c_bigger = c.size <=> d.size
if c.size > d.size
d += [nil] * (c.size - d.size)
elsif c.size < d.size
c += [nil] * (d.size - c.size)
end
c.zip(d) do |a, b|
return -1 if a.nil?
return 1 if b.nil?
if is_numeric(a) && is_numeric(b)
compare = a.to_big_i <=> b.to_big_i
return compare if compare != 0
else
compare = a <=> b
return compare if compare != 0
end
end
is_c_bigger
end
def compare_alphanumerically(a : String, b : String)
compare_alphanumerically split_by_alphanumeric(a), split_by_alphanumeric(b)
end

View File

@@ -7,6 +7,7 @@
<span hidden></span>
</span>
</li>
<li data-url="/admin/downloads">Download Manager</li>
</ul>
<hr class="uk-divider-icon">

View File

@@ -0,0 +1,32 @@
<div class="uk-margin">
<div id="actions" class="uk-margin">
<button class="uk-button uk-button-default" onclick="remove()">Delete Completed Tasks</button>
<button class="uk-button uk-button-default" onclick="refresh()">Retry Failed Tasks</button>
<button class="uk-button uk-button-default" onclick="load()">Refresh Queue</button>
<button class="uk-button uk-button-default" onclick="toggle()" id="pause-resume-btn" hidden></button>
</div>
<div id="config" class="uk-margin">
<label><input id="auto-refresh" class="uk-checkbox" type="checkbox" checked> Auto Refresh</label>
</div>
</div>
<table class="uk-table uk-table-striped uk-overflow-auto">
<thead>
<tr>
<th>Chapter</th>
<th>Manga</th>
<th>Progress</th>
<th>Time</th>
<th>Status</th>
<th>Actions</th>
</tr>
</thead>
</table>
<% content_for "script" do %>
<script>
var baseURL = "<%= base_url %>".replace(/\/$/, "");
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.24.0/moment.min.js"></script>
<script src="/js/alert.js"></script>
<script src="/js/download-manager.js"></script>
<% end %>

83
src/views/download.ecr Normal file
View File

@@ -0,0 +1,83 @@
<h2 class=uk-title>Download from MangaDex</h2>
<div class="uk-grid-small" uk-grid>
<div class="uk-width-3-4">
<input id="search-input" class="uk-input" type="text" placeholder="MangaDex manga ID or URL">
</div>
<div class="uk-width-1-4">
<div id="spinner" uk-spinner class="uk-align-center" hidden></div>
<button id="search-btn" class="uk-button uk-button-default" onclick="search()">Search</button>
</div>
</div>
<div class"uk-grid-small" uk-grid hidden id="manga-details">
<div class="uk-width-1-4@s">
<img id="cover">
</div>
<div class="uk-width-1-4@s">
<p id="title"></p>
<p id="artist"></p>
<p id="author"></p>
</div>
<div id="filter-form" class="uk-form-stacked uk-width-1-2@s" hidden>
<p class="uk-text-lead uk-margin-remove-bottom">Filter Chapters</p>
<p class="uk-text-meta uk-margin-remove-top" id="count-text"></p>
<div class="uk-margin">
<label class="uk-form-label" for="lang-select">Language</label>
<div class="uk-form-controls">
<select class="uk-select filter-field" id="lang-select">
</select>
</div>
</div>
<div class="uk-margin">
<label class="uk-form-label" for="group-select">Group</label>
<div class="uk-form-controls">
<select class="uk-select filter-field" id="group-select">
</select>
</div>
</div>
<div class="uk-margin">
<label class="uk-form-label" for="volume-range">Volume</label>
<div class="uk-form-controls">
<input class="uk-input filter-field" type="text" id="volume-range" placeholder="e.g., 127, 10-14, >30, <=212, or leave it empty.">
</div>
</div>
<div class="uk-margin">
<label class="uk-form-label" for="chapter-range">Chapter</label>
<div class="uk-form-controls">
<input class="uk-input filter-field" type="text" id="chapter-range" placeholder="e.g., 127, 10-14, >30, <=212, or leave it empty.">
</div>
</div>
</div>
</div>
<div id="selection-controls" class="uk-margin" hidden>
<div class="uk-margin">
<button class="uk-button uk-button-default" onclick="selectAll()">Select All</button>
<button class="uk-button uk-button-default" onclick="unselect()">Clear Selections</button>
<button class="uk-button uk-button-primary" id="download-btn" onclick="download()">Download Selected</button>
<div id="download-spinner" uk-spinner class="uk-margin-left" hidden></div>
</div>
<p class="uk-text-meta">Click on a table row to select the chapter. Drag your mouse over multiple rows to select them all. Hold Ctrl to make multiple non-adjacent selections.</p>
</div>
<p id="filter-notification" hidden></p>
<table class="uk-table uk-table-striped uk-overflow-auto" hidden>
<thead>
<tr>
<th>ID</th>
<th>Title</th>
<th>Language</th>
<th>Group</th>
<th>Volume</th>
<th>Chapter</th>
<th>Timestamp</th>
</tr>
</thead>
</table>
<% content_for "script" do %>
<script>
var baseURL = "<%= base_url %>".replace(/\/$/, "");
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.24.0/moment.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.12.1/jquery-ui.min.js"></script>
<script src="/js/alert.js"></script>
<script src="/js/download.js"></script>
<% end %>

View File

@@ -1,23 +1,43 @@
<h2 class=uk-title>Library</h2>
<p class="uk-text-meta"><%= titles.size %> titles found</p>
<div class="uk-margin">
<form class="uk-search uk-search-default">
<span uk-search-icon></span>
<input class="uk-search-input" type="search" placeholder="Search">
</form>
<div class="uk-grid-small" uk-grid>
<div class="uk-margin-bottom uk-width-3-4@s">
<form class="uk-search uk-search-default">
<span uk-search-icon></span>
<input class="uk-search-input" type="search" placeholder="Search">
</form>
</div>
<div class="uk-margin-bottom uk-width-1-4@s">
<div class="uk-form-horizontal">
<select class="uk-select" id="sort-select">
<option id="name-up">â–˛ Name</option>
<option id="name-down">â–Ľ Name</option>
<option id="date-up">â–˛ Date Modified</option>
<option id="date-down">â–Ľ Date Modified</option>
<option id="progress-up">â–˛ Progress</option>
<option id="progress-down">â–Ľ Progress</option>
</select>
</div>
</div>
</div>
<div class="uk-child-width-1-4@m uk-child-width-1-2" uk-grid>
<div id="item-container" class="uk-child-width-1-4@m uk-child-width-1-2" uk-grid>
<%- titles.each_with_index do |t, i| -%>
<div class="item">
<a class="acard" href="/book/<%= t.title %>">
<div class="item" data-mtime="<%= t.mtime.to_unix %>" data-progress="<%= percentage[i] %>">
<a class="acard" href="/book/<%= t.id %>">
<div class="uk-card uk-card-default">
<div class="uk-card-media-top">
<img src="<%= t.entries[0].cover_url %>" alt="">
<%- if t.entries.size > 0 -%>
<img data-src="<%= t.entries[0].cover_url %>" data-width data-height alt="" uk-img>
<%- else -%>
<img data-src="/img/icon.png" data-width data-height alt="" uk-img>
<%- end -%>
</div>
<div class="uk-card-body">
<%- if t.entries.size > 0 -%>
<div class="uk-card-badge uk-label"><%= (percentage[i] * 100).round(1) %>%</div>
<h3 class="uk-card-title"><%= t.title %></h3>
<p><%= t.entries.size %> entries</p>
<%- end -%>
<h3 class="uk-card-title break-word" data-title="<%= t.title.gsub("\"", "&quot;") %>"><%= t.title %></h3>
<p><%= t.size %> entries</p>
</div>
</div>
</a>
@@ -26,5 +46,8 @@
</div>
<% content_for "script" do %>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jQuery.dotdotdot/4.0.11/dotdotdot.js"></script>
<script src="/js/dots.js"></script>
<script src="/js/search.js"></script>
<script src="/js/sort-items.js"></script>
<% end %>

View File

@@ -8,6 +8,10 @@
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/uikit@3.3.1/dist/css/uikit.min.css" />
<link rel="stylesheet" href="/css/mango.css" />
<script defer src="/js/fontawesome.min.js"></script>
<script defer src="/js/solid.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
<script src="/js/theme.js"></script>
</head>
<body>
@@ -18,7 +22,9 @@
<ul class="uk-nav uk-nav-primary uk-nav-center uk-margin-auto-vertical">
<li><a href="/">Home</a></li>
<li><a href="/admin">Admin</a></li>
<li><a href="/download">Download</a></li>
<hr uk-divider>
<li><a onclick="toggleTheme()"><i class="fas fa-adjust"></i></a></li>
<li><a href="/logout">Logout</a></li>
</ul>
</div>
@@ -35,23 +41,28 @@
<ul class="uk-navbar-nav">
<li><a href="/">Home</a></li>
<li><a href="/admin">Admin</a></li>
<li><a href="/download">Download</a></li>
</ul>
</div>
<div class="uk-navbar-right uk-visible@s">
<ul class="uk-navbar-nav">
<li><a onclick="toggleTheme()"><i class="fas fa-adjust"></i></a></li>
<li><a href="/logout">Logout</a></li>
</ul>
</div>
</div>
</div>
<div class="uk-section uk-section-default uk-section-small">
<div class="uk-section uk-section-small">
</div>
<div class="uk-section uk-section-default uk-section-small">
<div class="uk-section uk-section-small">
<div class="uk-container uk-container-small">
<div id="alert"></div>
<%= content %>
</div>
</div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
<script>
setTheme(getTheme());
</script>
<script src="https://cdn.jsdelivr.net/npm/uikit@3.3.1/dist/js/uikit.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/uikit@3.3.1/dist/js/uikit-icons.min.js"></script>

View File

@@ -10,7 +10,8 @@
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/uikit@3.3.1/dist/css/uikit.min.css" />
</head>
<body>
<div class="uk-section uk-section-muted uk-flex uk-flex-middle uk-animation-fade" uk-height-viewport="">
<script src="/js/theme.js"></script>
<div class="uk-section uk-flex uk-flex-middle uk-animation-fade" uk-height-viewport="">
<div class="uk-width-1-1">
<div class="uk-container">
<div class="uk-grid-margin uk-grid uk-grid-stack" uk-grid="">

View File

@@ -1 +1 @@
<p class="uk-text-lead"><%= message %></p>
<p class="uk-text-lead uk-text-center"><%= message %></p>

View File

@@ -11,6 +11,7 @@
</head>
<body>
<script src="/js/theme.js"></script>
<div class="uk-section uk-section-default uk-section-small reader-bg">
<div class="uk-container uk-container-small">
<%- urls.each_with_index do |url, i| -%>
@@ -36,6 +37,9 @@
<h3 class="uk-modal-title">Options</h3>
</div>
<div class="uk-modal-body">
<div class="uk-margin">
<p id="progress-label"></p>
</div>
<div class="uk-margin">
<label class="uk-form-label" for="form-stacked-select">Jump to page</label>
<div class="uk-form-controls">

View File

@@ -1,23 +1,65 @@
<div id="alert"></div>
<h2 class=uk-title><%= title.title %></h2>
<p class="uk-text-meta"><%= title.entries.size %> entries found</p>
<div class="uk-margin">
<form class="uk-search uk-search-default">
<span uk-search-icon></span>
<input class="uk-search-input" type="search" placeholder="Search">
</form>
<ul class="uk-breadcrumb">
<li><a href="/">Library</a></li>
<%- title.parents.each do |t| -%>
<li><a href="/book/<%= t.id %>"><%= t.title %></a></li>
<%- end -%>
<li class="uk-disabled"><a><%= title.title %></a></li>
</ul>
<p class="uk-text-meta"><%= title.size %> entries found</p>
<div class="uk-grid-small" uk-grid>
<div class="uk-margin-bottom uk-width-3-4@s">
<form class="uk-search uk-search-default">
<span uk-search-icon></span>
<input class="uk-search-input" type="search" placeholder="Search">
</form>
</div>
<div class="uk-margin-bottom uk-width-1-4@s">
<div class="uk-form-horizontal">
<select class="uk-select" id="sort-select">
<option id="auto-up">â–˛ Auto</option>
<option id="auto-down">â–Ľ Auto</option>
<option id="name-up">â–˛ Name</option>
<option id="name-down">â–Ľ Name</option>
<option id="date-up">â–˛ Date Modified</option>
<option id="date-down">â–Ľ Date Modified</option>
<option id="progress-up">â–˛ Progress</option>
<option id="progress-down">â–Ľ Progress</option>
</select>
</div>
</div>
</div>
<div class="uk-child-width-1-4@m uk-child-width-1-2" uk-grid>
<div id="item-container" class="uk-child-width-1-4@m uk-child-width-1-2" uk-grid>
<%- title.titles.each_with_index do |t, i| -%>
<div class="item" data-mtime="<%= t.mtime.to_unix %>" data-progress="0.0">
<a class="acard" href="/book/<%= t.id %>">
<div class="uk-card uk-card-default">
<div class="uk-card-media-top">
<%- if t.entries.size > 0 -%>
<img data-src="<%= t.entries[0].cover_url %>" data-width data-height alt="" uk-img>
<%- else -%>
<img data-src="/img/icon.png" data-width data-height alt="" uk-img>
<%- end -%>
</div>
<div class="uk-card-body">
<h3 class="uk-card-title break-word" data-title="<%= t.title.gsub("\"", "&quot;") %>"><%= t.title %></h3>
<p><%= t.size %> entries</p>
</div>
</div>
</a>
</div>
<%- end -%>
<%- title.entries.each_with_index do |e, i| -%>
<div class="item">
<div class="item" data-mtime="<%= e.mtime.to_unix %>" data-progress="<%= percentage[i] %>">
<a class="acard">
<div class="uk-card uk-card-default" onclick="showModal('<%= e.title %>', '<%= e.zip_path %>', '<%= e.pages %>', <%= (percentage[i] * 100).round(1) %>, '<%= title.title %>', '<%= e.title %>')">
<div class="uk-card uk-card-default" onclick="showModal(&quot;<%= e.encoded_path %>&quot;, '<%= e.pages %>', <%= (percentage[i] * 100).round(1) %>, &quot;<%= title.encoded_title %>&quot;, &quot;<%= e.encoded_title %>&quot;, '<%= e.title_id %>', '<%= e.id %>')">
<div class="uk-card-media-top">
<img src="<%= e.cover_url %>" alt="">
<img data-src="<%= e.cover_url %>" alt="" data-width data-height uk-img>
</div>
<div class="uk-card-body">
<div class="uk-card-badge uk-label"><%= (percentage[i] * 100).round(1) %>%</div>
<h3 class="uk-card-title"><%= e.title %></h3>
<h3 class="uk-card-title break-word" data-title="<%= e.title.gsub("\"", "&quot;") %>"><%= e.title %></h3>
<p><%= e.pages %> pages</p>
</div>
</div>
</a>
@@ -29,8 +71,8 @@
<div class="uk-modal-dialog uk-margin-auto-vertical">
<button class="uk-modal-close-default" type="button" uk-close></button>
<div class="uk-modal-header">
<h3 class="uk-modal-title" id="modal-title"></h3>
<p class="uk-text-meta uk-margin-remove-bottom" id="path-text"></p>
<h3 class="uk-modal-title break-word" id="modal-title"></h3>
<p class="uk-text-meta uk-margin-remove-bottom break-word" id="path-text"></p>
<p class="uk-text-meta uk-margin-remove-top" id="pages-text"></p>
</div>
<div class="uk-modal-body">
@@ -49,6 +91,10 @@
</div>
<% content_for "script" do %>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jQuery.dotdotdot/4.0.11/dotdotdot.js"></script>
<script src="/js/dots.js"></script>
<script src="/js/alert.js"></script>
<script src="/js/title.js"></script>
<script src="/js/search.js"></script>
<script src="/js/sort-items.js"></script>
<% end %>

View File

@@ -1,5 +1,3 @@
<div id="alert"></div>
<form action="/admin/user/edit" method="post" accept-charset="utf-8">
<div class="uk-margin">
@@ -51,5 +49,6 @@
error = '<%= error %>';
<%- end -%>
</script>
<script src="/js/alert.js"></script>
<script src="/js/user-edit.js"></script>
<% end %>

View File

@@ -1,4 +1,3 @@
<div id="alert"></div>
<table class="uk-table uk-table-divider">
<thead>
<tr>
@@ -27,5 +26,6 @@
<% content_for "script" do %>
<script src="/js/alert.js"></script>
<script src="/js/user.js"></script>
<% end %>