From 456878ba06358a77d4ab9312fdc69bf780f8fdf4 Mon Sep 17 00:00:00 2001 From: KOBAYASHI Shuji Date: Sun, 13 Dec 2020 15:27:53 +0900 Subject: Improve source scanning for presym The accuracy is greatly improved by using the C preprocessor to scan C sources for presym. C preprocessor can perfectly interpret all comments and preprocessor directives, so it can detect all symbols defined, for example `mrbgems/mruby-socket/src/const.cstub`. Also, as described later, this change will greatly improve the accuracy of presym detection from Ruby sources. ## Result The number of lines in the `presym` file for all gems is as follows: ```console Previous: 999 (false positive = 89, undetected = 297) New: 1207 ``` ## Build process The new build process (with presym) is as follows: 1. Build `mrbc` without presym (more on building without presym later). 2. Compile Ruby sources to C struct format with the `mrbc` created in step 1, and create` mrblib.c` and `gem_init.c`. Note that the symbols in the created files are output as `MRB_SYM` family macros or `mrb_intern_lit` instead of IDs (details will be described later). 3. C preprocessor processes C sources including the created files of step 2 and outputs them as `.i` files. In these files, for example, `MRB_IVSYM(foo)` is converted to `<@! "@" "foo" !@>` and `mrb_define_module(mrb, "Foo")` is converted to `<@! "Foo" !@>`. 4. Scan the files created in step 3 and create `presym` and` presym.inc` files. The files created in step 2 should output all static symbols defined in Ruby sources, including local variables, so we can detect all presyms by just scanning C sources without scanning Ruby sources directly. Further, by this process, the files to be scanned becomes the same as the files to be compiled, so that there is no excess or deficiency. ## Related changes The following changes have been made in relation to realizing this feature. ### Allow build without presym It enables build without presym to achieve the "Build process: 1". This incorporates #5202, see its issue for details. Note that when presym is enabled, even adding a local variable to a Ruby source may change contents of presym and require recompilation of almost all C sources. This is inconvenient, especially during trial and error in development, but this feature is also useful because it does not cause this problem if presym is disabled. ### Automatically create build target for `mrbc` without presym The `mrbc` used in the "Build process: 1" will be built by automatically creating a build target for it. The build name is `SOURCE_BUILD_NAME/mrbc`. ### Constantize output of C struct format by `mrbc` To realizing the "Build process: 2", as mentioned above, symbol IDs are not output directly in C struct format output by `mrbc`. As a result, the output becomes constant regardless of the state of presym at the time of `mrbc` build, and it is possible to detect symbols of Ruby sources in the same way as other C sources. Note that `mrb_intern_lit` is used for symbols that do not become presym, but in this state, the corresponding element in the symbol array cannot be statically initialized, so it is initialized at run time (therefore, in this case, the `const` qualifier is not added to the symbol array). ### Specify arbitrary `mrbc` file To realizing the "Build process: 2", enabled to specify `mrbc` created by another build target or pre-built` mrbc`. Use `MRuby::Build#mrbcfile =` to specify it explicitly. You can omit the "Build process: 1" by specifying pre-built `mrbc`, and you can always use an optimized build to compile Ruby sources faster. I think changes that affect the output of `mrbc` are rare, so in many cases it helps to improve efficiency. With presym, the build will be a little slower due to more build steps, but this feature will improve it a bit. ### Create presym files for each build target This feature was proposed at #5194 and merged once, but was reverted in 5c205e6e due to problems especially with cross-compilation. It has been introduced again because this change solves the problem. The presym files will be created below. * `build/NAME/presym` * `build/NAME/include/mruby/presym.inc` ### Other changes * Because presym detection accuracy is greatly improved as mentioned above, `MRuby::Gem::Specification#cdump?` is set to true by default, and `disable_cdump` is added instead of `enable_cdump`. Also, support for gem specific presym files has been discontinued (https://github.com/mruby/mruby/issues/5151#issuecomment-730967232). * Previously, `mrbc` was automatically created for the `host` build, but it will not be created if the build target for `mrbc` mentioned above is automatically created. At this time, `mrbc` file of the `mrbc` build is copied to` bin/`. * Two types of `.d` files will be created, `.o.d` and `.i.d`. oThis is because if `.i` depends on `presym.inc`, the dependency will circulate, so the `.d` file cannot be shared. * Changed file created with `enable_cxx_exception` to `X-cxx.cxx` from `X.cxx` to use the mruby standard Rake rule. ### Note Almost all C sources will need to be recompiled if there are any changes to `persym.inc` (if not recompiled properly, it will often result in run-time error). If `gcc` toolchain is used, dependencies are resolved by the `.d` file, so it become automatically recompile target, but if not (e.g. MSVC), it is necessary to manually make it recompile target. Also, even if `gcc` toolchain is used, it may not become recompile target if external gems does not use the mruby standard Rake rule. In particular, if the standard rule is overwritten, such as https://github.com/mruby/mruby/pull/5112/files, `.d` file will not be read, so be careful. --- lib/mruby/build.rb | 147 +++++++++++++++++++++++++++++++++++++-------- lib/mruby/build/command.rb | 34 ++++++++--- lib/mruby/gem.rb | 33 +++++----- lib/mruby/presym.rb | 113 ++++++++++++++++++++++++++++++++++ 4 files changed, 277 insertions(+), 50 deletions(-) create mode 100644 lib/mruby/presym.rb (limited to 'lib') diff --git a/lib/mruby/build.rb b/lib/mruby/build.rb index 2e67fe5a6..484613964 100644 --- a/lib/mruby/build.rb +++ b/lib/mruby/build.rb @@ -5,6 +5,7 @@ require "mruby/build/command" module MRuby autoload :Gem, "mruby/gem" autoload :Lockfile, "mruby/lockfile" + autoload :Presym, "mruby/presym" class << self def targets @@ -39,6 +40,7 @@ module MRuby class Build class << self attr_accessor :current + def mruby_config_path path = ENV['MRUBY_CONFIG'] || ENV['CONFIG'] if path.nil? || path.empty? @@ -49,11 +51,16 @@ module MRuby end path end + + def install_dir + @install_dir ||= ENV['INSTALL_DIR'] || "#{MRUBY_ROOT}/bin" + end end + include Rake::DSL include LoadGems attr_accessor :name, :bins, :exts, :file_separator, :build_dir, :gem_clone_dir - attr_reader :libmruby_core_objs, :libmruby_objs, :gems, :toolchains, :gem_dir_to_repo_url + attr_reader :products, :libmruby_core_objs, :libmruby_objs, :gems, :toolchains, :presym, :mrbc_build, :gem_dir_to_repo_url alias libmruby libmruby_objs @@ -61,16 +68,16 @@ module MRuby COMMANDS = COMPILERS + %w(linker archiver yacc gperf git exts mrbc) attr_block MRuby::Build::COMMANDS - Exts = Struct.new(:object, :executable, :library) + Exts = Struct.new(:object, :executable, :library, :preprocessed) - def initialize(name='host', build_dir=nil, &block) + def initialize(name='host', build_dir=nil, internal: false, &block) @name = name.to_s unless current = MRuby.targets[@name] if ENV['OS'] == 'Windows_NT' - @exts = Exts.new('.o', '.exe', '.a') + @exts = Exts.new('.o', '.exe', '.a', '.i') else - @exts = Exts.new('.o', '', '.a') + @exts = Exts.new('.o', '', '.a', '.i') end build_dir = build_dir || ENV['MRUBY_BUILD_DIR'] || "#{MRUBY_ROOT}/build" @@ -89,6 +96,7 @@ module MRuby @git = Command::Git.new(self) @mrbc = Command::Mrbc.new(self) + @products = [] @bins = [] @gems = MRuby::Gem::List.new @libmruby_core_objs = [] @@ -101,6 +109,9 @@ module MRuby @enable_bintest = false @enable_test = false @enable_lock = true + @enable_presym = true + @mrbcfile_external = false + @internal = internal @toolchains = [] @gem_dir_to_repo_url = {} @@ -108,7 +119,14 @@ module MRuby end current.instance_eval(&block) - current.build_mrbc_exec if current.libmruby_enabled? && @name == "host" + if current.libmruby_enabled? && !current.mrbcfile_external? + if current.presym_enabled? + current.create_mrbc_build if current.host? || current.gems["mruby-bin-mrbc"] + elsif current.host? + current.build_mrbc_exec + end + end + current.presym = Presym.new(current) if current.presym_enabled? current.build_mrbtest if current.test_enabled? end @@ -136,6 +154,17 @@ module MRuby @enable_debug = true end + def presym_enabled? + @enable_presym + end + + def disable_presym + if @enable_presym + @enable_presym = false + compilers.each{|c| c.defines << "MRB_NO_PRESYM"} + end + end + def disable_lock @enable_lock = false end @@ -187,8 +216,29 @@ module MRuby @cxx_abi_enabled = true end - def compile_as_cxx src, cxx_src, obj = nil, includes = [] - obj = objfile(cxx_src) if obj.nil? + def compile_as_cxx(src, cxx_src = nil, obj = nil, includes = []) + # + # If `cxx_src` is specified, this method behaves the same as before as + # compatibility mode, but `.d` file is not read. + # + # If `cxx_src` is omitted, `.d` file is read by using mruby standard + # Rake rule (C++ source name is also changed). + # + if cxx_src + obj ||= cxx_src + @exts.object + dsts = [obj] + dsts << (cxx_src + @exts.preprocessed) if presym_enabled? + defines = [] + include_paths = ["#{MRUBY_ROOT}/src", *includes] + dsts.each do |dst| + file dst => cxx_src do |t| + cxx.run t.name, t.prerequisites.first, defines, include_paths + end + end + else + cxx_src = "#{build_dir}/#{src.relative_path})".ext << "-cxx.cxx" + obj = cxx_src.ext(@exts.object) + end file cxx_src => [src, __FILE__] do |t| mkdir_p File.dirname t.name @@ -206,10 +256,6 @@ extern "C" { EOS end - file obj => cxx_src do |t| - cxx.run t.name, t.prerequisites.first, [], ["#{MRUBY_ROOT}/src"] + includes - end - obj end @@ -250,11 +296,11 @@ EOS end def build_mrbtest - gem :core => 'mruby-test' + gem :core => 'mruby-test' unless @gems['mruby-test'] end def build_mrbc_exec - gem :core => 'mruby-bin-mrbc' + gem :core => 'mruby-bin-mrbc' unless @gems['mruby-bin-mrbc'] end def locks @@ -265,10 +311,23 @@ EOS return @mrbcfile if @mrbcfile gem_name = "mruby-bin-mrbc" - gem = gems[gem_name] || MRuby.targets["host"].gems[gem_name] + gem = @gems[gem_name] + gem ||= (host = MRuby.targets["host"]) && host.gems[gem_name] + unless gem + fail "external mrbc or mruby-bin-mrbc gem in current('#{@name}') or 'host' build is required" + end @mrbcfile = exefile("#{gem.build.build_dir}/bin/mrbc") end + def mrbcfile=(path) + @mrbcfile = path + @mrbcfile_external = true + end + + def mrbcfile_external? + @mrbcfile_external + end + def compilers COMPILERS.map do |c| instance_variable_get("@#{c}") @@ -276,7 +335,7 @@ EOS end def define_rules - use_mrdb = @gems.find{|g| g.name == "mruby-bin-debugger"} + use_mrdb = @gems["mruby-bin-debugger"] compilers.each do |compiler| if respond_to?(:enable_gems?) && enable_gems? compiler.defines -= %w(MRB_NO_GEMS) @@ -285,7 +344,10 @@ EOS end compiler.defines |= %w(MRB_USE_DEBUG_HOOK) if use_mrdb end - cc.define_rules(build_dir, MRUBY_ROOT) + [@cc, *(@cxx if cxx_exception_enabled?)].each do |compiler| + compiler.define_rules(@build_dir, MRUBY_ROOT, @exts.object) + compiler.define_rules(@build_dir, MRUBY_ROOT, @exts.preprocessed) if presym_enabled? + end end def filename(name) @@ -346,7 +408,8 @@ EOS puts ">>> Bintest #{name} <<<" targets = @gems.select { |v| File.directory? "#{v.dir}/bintest" }.map { |v| filename v.dir } targets << filename(".") if File.directory? "./bintest" - env = {"BUILD_DIR" => @build_dir} + mrbc = @gems["mruby-bin-mrbc"] ? exefile("#{@build_dir}/bin/mrbc") : mrbcfile + env = {"BUILD_DIR" => @build_dir, "MRBCFILE" => mrbc} sh env, "ruby test/bintest.rb#{verbose_flag} #{targets.join ' '}" end @@ -380,6 +443,41 @@ EOS def libraries [libmruby_static] end + + def host? + @name == "host" + end + + def internal? + @internal + end + + protected + + attr_writer :presym + + def create_mrbc_build + exclusions = %i[@name @build_dir @gems @enable_test @enable_bintest @internal] + name = "#{@name}/mrbc" + MRuby.targets.delete(name) + build = self.class.new(name, internal: true){} + instance_variables.each do |n| + next if exclusions.include?(n) + v = instance_variable_get(n) + v = case v + when nil, true, false, Numeric; v + when String, Command; v.clone + else Marshal.load(Marshal.dump(v)) # deep clone + end + build.instance_variable_set(n, v) + end + build.build_mrbc_exec + build.disable_libmruby + build.disable_presym + @mrbc_build = build + self.mrbcfile = build.mrbcfile + build + end end # Build class CrossBuild < Build @@ -392,7 +490,7 @@ EOS def initialize(name, build_dir=nil, &block) @test_runner = Command::CrossTestRunner.new(self) super - unless MRuby.targets['host'] + unless mrbcfile_external? || MRuby.targets['host'] # add minimal 'host' MRuby::Build.new('host') do |conf| if ENV['VisualStudioVersion'] || ENV['VSINSTALLDIR'] @@ -400,16 +498,13 @@ EOS else toolchain :gcc end - conf.gem :core => 'mruby-bin-mrbc' + conf.build_mrbc_exec conf.disable_libmruby + conf.disable_presym end end end - def mrbcfile - MRuby.targets['host'].mrbcfile - end - def run_test @test_runner.runner_options << verbose_flag mrbtest = exefile("#{build_dir}/bin/mrbtest") @@ -420,5 +515,9 @@ EOS @test_runner.run(mrbtest) end end + + protected + + def create_mrbc_build; end end # CrossBuild end # MRuby diff --git a/lib/mruby/build/command.rb b/lib/mruby/build/command.rb index 80657eadc..9362a9d95 100644 --- a/lib/mruby/build/command.rb +++ b/lib/mruby/build/command.rb @@ -42,6 +42,7 @@ module MRuby attr_accessor :label, :flags, :include_paths, :defines, :source_exts attr_accessor :compile_options, :option_define, :option_include_path, :out_ext attr_accessor :cxx_compile_flag, :cxx_exception_flag, :cxx_invalid_flags + attr_writer :preprocess_options def initialize(build, source_exts=[], label: "CC") super(build) @@ -55,9 +56,15 @@ module MRuby @option_define = %q[-D"%s"] @compile_options = %q[%{flags} -o "%{outfile}" -c "%{infile}"] @cxx_invalid_flags = [] + @out_ext = build.exts.object end alias header_search_paths include_paths + + def preprocess_options + @preprocess_options ||= @compile_options.sub(/(?:\A|\s)\K-c(?=\s)/, "-E -P") + end + def search_header_path(name) header_search_paths.find do |v| File.exist? build.filename("#{v}/#{name}").sub(/^"(.*)"$/, '\1') @@ -79,13 +86,20 @@ module MRuby def run(outfile, infile, _defines=[], _include_paths=[], _flags=[]) mkdir_p File.dirname(outfile) - _pp @label, infile.relative_path, outfile.relative_path - _run compile_options, { :flags => all_flags(_defines, _include_paths, _flags), - :infile => filename(infile), :outfile => filename(outfile) } + flags = all_flags(_defines, _include_paths, _flags) + if File.extname(outfile) == build.exts.object + label = @label + opts = compile_options + else + label = "CPP" + opts = preprocess_options + flags << " -DMRB_PRESYM_SCANNING" + end + _pp label, infile.relative_path, outfile.relative_path + _run opts, flags: flags, infile: filename(infile), outfile: filename(outfile) end - def define_rules(build_dir, source_dir='') - @out_ext = build.exts.object + def define_rules(build_dir, source_dir='', out_ext=build.exts.object) gemrake = File.join(source_dir, "mrbgem.rake") rakedep = File.exist?(gemrake) ? [ gemrake ] : [] @@ -143,10 +157,10 @@ module MRuby # /src/value_array.h: # def get_dependencies(file) - file = file.ext('d') unless File.extname(file) == '.d' - return [MRUBY_CONFIG] unless File.exist?(file) + dep_file = "#{file}.d" + return [MRUBY_CONFIG] unless File.exist?(dep_file) - deps = File.read(file).gsub("\\\n ", "").split("\n").map do |dep_line| + deps = File.read(dep_file).gsub("\\\n ", "").split("\n").map do |dep_line| # dep_line: # - "/build/host/src/array.o: /src/array.c /include/mruby/common.h ..." # - "" @@ -187,6 +201,10 @@ module MRuby [libraries, _libraries].flatten.map{ |d| option_library % d }.join(' ') end + def run_attrs + [@libraries, @library_paths, @flags, @flags_before_libraries, @flags_after_libraries] + end + def run(outfile, objfiles, _libraries=[], _library_paths=[], _flags=[], _flags_before_libraries=[], _flags_after_libraries=[]) mkdir_p File.dirname(outfile) library_flags = [libraries, _libraries].flatten.map { |d| option_library % d } diff --git a/lib/mruby/gem.rb b/lib/mruby/gem.rb index e3b758f4f..4e918745f 100644 --- a/lib/mruby/gem.rb +++ b/lib/mruby/gem.rb @@ -7,7 +7,6 @@ module MRuby class << self attr_accessor :current end - LinkerConfig = Struct.new(:libraries, :library_paths, :flags, :flags_before_libraries, :flags_after_libraries) class Specification include Rake::DSL @@ -50,13 +49,13 @@ module MRuby end def setup - return if defined?(@linker) # return if already set up + return if defined?(@bins) # return if already set up MRuby::Gem.current = self MRuby::Build::COMMANDS.each do |command| instance_variable_set("@#{command}", @build.send(command).clone) end - @linker = LinkerConfig.new([], [], [], [], []) + @linker.run_attrs.each(&:clear) @rbfiles = Dir.glob("#{@dir}/mrblib/**/*.rb").sort @objs = Dir.glob("#{@dir}/src/*.{c,cpp,cxx,cc,m,asm,s,S}").map do |f| @@ -72,6 +71,7 @@ module MRuby @test_args = {} @bins = [] + @cdump = true @requirements = [] @export_include_paths = [] @@ -81,7 +81,6 @@ module MRuby @generate_functions = !(@rbfiles.empty? && @objs.empty?) @objs << objfile("#{build_dir}/gem_init") if @generate_functions - @cdump = core? # by default core gems use cdump and others use mrb dump if !name || !licenses || !authors fail "#{name || dir} required to set name, license(s) and author(s)" @@ -96,10 +95,11 @@ module MRuby end def setup_compilers - (core? ? [cc] : compilers).each do |compiler| - compiler.define_rules build_dir, "#{dir}" + (core? ? [@cc, *(@cxx if build.cxx_exception_enabled?)] : compilers).each do |compiler| + compiler.define_rules build_dir, @dir, @build.exts.preprocessed if build.presym_enabled? + compiler.define_rules build_dir, @dir, @build.exts.object compiler.defines << %Q[MRBGEM_#{funcname.upcase}_VERSION=#{version}] - compiler.include_paths << "#{dir}/include" if File.directory? "#{dir}/include" + compiler.include_paths << "#{@dir}/include" if File.directory? "#{@dir}/include" end define_gem_init_builder if @generate_functions @@ -114,12 +114,12 @@ module MRuby return false end - def enable_cdump - @cdump = true + def disable_cdump + @cdump = false end def cdump? - @cdump + build.presym_enabled? && @cdump end def core? @@ -175,7 +175,6 @@ module MRuby end def define_gem_init_builder - file objfile("#{build_dir}/gem_init") => [ "#{build_dir}/gem_init.c", File.join(dir, "mrbgem.rake") ] file "#{build_dir}/gem_init.c" => [build.mrbcfile, __FILE__] + [rbfiles].flatten do |t| mkdir_p build_dir generate_gem_init("#{build_dir}/gem_init.c") @@ -187,7 +186,7 @@ module MRuby open(fname, 'w') do |f| print_gem_init_header f unless rbfiles.empty? - if @cdump + if cdump? build.mrbc.run f, rbfiles, "gem_mrblib_#{funcname}_proc" else build.mrbc.run f, rbfiles, "gem_mrblib_irep_#{funcname}", false @@ -198,10 +197,10 @@ module MRuby f.puts %Q[] f.puts %Q[void GENERATED_TMP_mrb_#{funcname}_gem_init(mrb_state *mrb) {] f.puts %Q[ int ai = mrb_gc_arena_save(mrb);] - f.puts %Q[ struct REnv *e;] unless rbfiles.empty? + f.puts %Q[ gem_mrblib_#{funcname}_proc_init_syms(mrb);] if !rbfiles.empty? && cdump? f.puts %Q[ mrb_#{funcname}_gem_init(mrb);] if objs != [objfile("#{build_dir}/gem_init")] unless rbfiles.empty? - if @cdump + if cdump? f.puts %Q[ mrb_load_proc(mrb, gem_mrblib_#{funcname}_proc);] else f.puts %Q[ mrb_load_irep(mrb, gem_mrblib_irep_#{funcname});] @@ -211,7 +210,7 @@ module MRuby f.puts %Q[ mrb_close(mrb);] f.puts %Q[ exit(EXIT_FAILURE);] f.puts %Q[ }] - f.puts %Q[ e = mrb->c->cibase->env;] + f.puts %Q[ struct REnv *e = mrb->c->cibase->env;] f.puts %Q[ mrb->c->cibase->env = NULL;] f.puts %Q[ mrb_env_unshare(mrb, e);] end @@ -241,8 +240,6 @@ module MRuby f.puts %Q[#include ] else f.puts %Q[#include ] - f.puts %Q[#include ] - f.puts %Q[#include ] end end @@ -251,7 +248,7 @@ module MRuby f.puts %Q[#include ] f.puts %Q[#include ] f.puts %Q[#include ] - f.puts %Q[#include ] + f.puts %Q[#include ] f.puts %Q[#include ] f.puts %Q[#include ] unless test_args.empty? end diff --git a/lib/mruby/presym.rb b/lib/mruby/presym.rb new file mode 100644 index 000000000..75a903dba --- /dev/null +++ b/lib/mruby/presym.rb @@ -0,0 +1,113 @@ +module MRuby + class Presym + include Rake::DSL + + OPERATORS = { + "!" => "not", + "%" => "mod", + "&" => "and", + "*" => "mul", + "+" => "add", + "-" => "sub", + "/" => "div", + "<" => "lt", + ">" => "gt", + "^" => "xor", + "`" => "tick", + "|" => "or", + "~" => "neg", + "!=" => "neq", + "!~" => "nmatch", + "&&" => "andand", + "**" => "pow", + "+@" => "plus", + "-@" => "minus", + "<<" => "lshift", + "<=" => "le", + "==" => "eq", + "=~" => "match", + ">=" => "ge", + ">>" => "rshift", + "[]" => "aref", + "||" => "oror", + "<=>" => "cmp", + "===" => "eqq", + "[]=" => "aset", + }.freeze + + SYMBOL_TO_MACRO = { + # Symbol => Macro + # [prefix, suffix] => [prefix, suffix] + ["@@" , "" ] => ["CV" , "" ], + ["@" , "" ] => ["IV" , "" ], + ["" , "!" ] => ["" , "_B" ], + ["" , "?" ] => ["" , "_Q" ], + ["" , "=" ] => ["" , "_E" ], + ["" , "" ] => ["" , "" ], + }.freeze + + C_STR_LITERAL_RE = /"(?:[^\\\"]|\\.)*"/ + + def initialize(build) + @build = build + end + + def scan(paths) + presym_hash = {} + paths.each {|path| read_preprocessed(presym_hash, path)} + presym_hash.keys.sort_by!{|sym| [c_literal_size(sym), sym]} + end + + def read_list + File.readlines(list_path, mode: "r:binary").each(&:chomp!) + end + + def write_list(presyms) + _pp "GEN", list_path.relative_path + File.binwrite(list_path, presyms.join("\n") << "\n") + end + + def write_header(presyms) + prefix_re = Regexp.union(*SYMBOL_TO_MACRO.keys.map(&:first).uniq) + suffix_re = Regexp.union(*SYMBOL_TO_MACRO.keys.map(&:last).uniq) + sym_re = /\A(#{prefix_re})?([\w&&\D]\w*)(#{suffix_re})?\z/o + _pp "GEN", header_path.relative_path + mkdir_p(File.dirname(header_path)) + File.open(header_path, "w:binary") do |f| + f.puts "/* MRB_PRESYM_NAMED(lit, num, type, name) */" + f.puts "/* MRB_PRESYM_UNNAMED(lit, num) */" + presyms.each.with_index(1) do |sym, num| + if sym_re =~ sym && (affixes = SYMBOL_TO_MACRO[[$1, $3]]) + f.puts %|MRB_PRESYM_NAMED("#{sym}", #{num}, #{affixes * 'SYM'}, #{$2})| + elsif name = OPERATORS[sym] + f.puts %|MRB_PRESYM_NAMED("#{sym}", #{num}, OPSYM, #{name})| + elsif + f.puts %|MRB_PRESYM_UNNAMED("#{sym}", #{num})| + end + end + f.puts "#define MRB_PRESYM_MAX #{presyms.size}" + end + end + + def list_path + @list_pat ||= "#{@build.build_dir}/presym".freeze + end + + def header_path + @header_path ||= "#{@build.build_dir}/include/mruby/presym.inc".freeze + end + + private + + def read_preprocessed(presym_hash, path) + File.binread(path).scan(/<@! (.*?) !@>/) do |part,| + literals = part.scan(C_STR_LITERAL_RE) + presym_hash[literals.map{|l| l[1..-2]}.join] = true unless literals.empty? + end + end + + def c_literal_size(literal_without_quote) + literal_without_quote.size # TODO: consider escape sequence + end + end +end -- cgit v1.2.3